[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:11:25 dist-test-slave-68xk krb5kdc[10945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:11:25 dist-test-slave-68xk krb5kdc[10945](info): set up 2 sockets
Aug 11 22:11:25 dist-test-slave-68xk krb5kdc[10945](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 22:11:26.957674 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.566s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:11:27 dist-test-slave-68xk krb5kdc[10945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:11:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:11:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:11:27.166939 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:39285
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:39285
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:27.609886 10965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:27.610428 10965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:27.610651 10965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:27.618342 10965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:11:27.618536 10965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:27.618649 10965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:27.618805 10965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:11:27.618907 10965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:11:27.629356 10965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:39285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:39285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:27.631222 10965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:27.633072 10965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:27.653757 10974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:27.654451 10972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:27.654682 10977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:27.655339 10965 server_base.cc:1034] running on GCE node
I20260811 22:11:27.656503 10965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:27.658814 10965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:27.660101 10965 hybrid_clock.cc:648] HybridClock initialized: now 1786486287659975 us; error 101 us; skew 500 ppm
I20260811 22:11:27.660667 10965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:27 dist-test-slave-68xk krb5kdc[10945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486287, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:27.664306 10965 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:11:27.666917 10965 webserver.cc:460] Webserver started at http://127.10.173.190:41203/ using document root <none> and password file <none>
I20260811 22:11:27.668061 10965 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:27.668236 10965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:27.668702 10965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:27.673035 10965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "c17e7d1b83384bc689066905288608fa"
format_stamp: "Formatted at 2026-08-11 22:11:27 on dist-test-slave-68xk"
server_key: "26fc2d9c775f966be2cb9050f169f707"
server_key_iv: "94f4cb1d70f4bb4561ef6651356284ea"
server_key_version: "encryptionkey@0"
I20260811 22:11:27.674058 10965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "c17e7d1b83384bc689066905288608fa"
format_stamp: "Formatted at 2026-08-11 22:11:27 on dist-test-slave-68xk"
server_key: "26fc2d9c775f966be2cb9050f169f707"
server_key_iv: "94f4cb1d70f4bb4561ef6651356284ea"
server_key_version: "encryptionkey@0"
I20260811 22:11:27.682619 10965 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260811 22:11:27.688815 10985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:27.690939 10965 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:11:27.691265 10965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "c17e7d1b83384bc689066905288608fa"
format_stamp: "Formatted at 2026-08-11 22:11:27 on dist-test-slave-68xk"
server_key: "26fc2d9c775f966be2cb9050f169f707"
server_key_iv: "94f4cb1d70f4bb4561ef6651356284ea"
server_key_version: "encryptionkey@0"
I20260811 22:11:27.691517 10965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:27.707944 10965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:27.713665 10965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:27.714121 10965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:27.741742 10965 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:39285
I20260811 22:11:27.741770 11057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:39285 every 8 connection(s)
I20260811 22:11:27.743960 10965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:11:27.748889 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 10965
I20260811 22:11:27.749297 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:11:27.751219 11058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:27.751555 10934 external_mini_cluster.cc:1428] Setting key 0cd607b65d75bc41c8e1ba7adb43dd2d
I20260811 22:11:27.767123 11058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa: Bootstrap starting.
I20260811 22:11:27.774530 11058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:27.776639 11058 log.cc:824] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:27.781261 11058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa: No bootstrap required, opened a new log
Aug 11 22:11:27 dist-test-slave-68xk krb5kdc[10945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:27.795188 11058 raft_consensus.cc:348] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17e7d1b83384bc689066905288608fa" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39285 } }
I20260811 22:11:27.795830 11058 raft_consensus.cc:374] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:27.796059 11058 raft_consensus.cc:729] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17e7d1b83384bc689066905288608fa, State: Initialized, Role: FOLLOWER
I20260811 22:11:27.797468 11058 consensus_queue.cc:260] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17e7d1b83384bc689066905288608fa" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39285 } }
I20260811 22:11:27.798012 11058 raft_consensus.cc:388] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:11:27.798437 11058 raft_consensus.cc:482] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:11:27.798744 11058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:27.801534 11058 raft_consensus.cc:504] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17e7d1b83384bc689066905288608fa" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39285 } }
I20260811 22:11:27.802606 11058 leader_election.cc:302] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c17e7d1b83384bc689066905288608fa; no voters:
I20260811 22:11:27.804042 11058 leader_election.cc:288] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:11:27.804328 11066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:27.805106 11061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:27.756963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52800 (local address 127.10.173.190:39285)
0811 22:11:27.758184 (+ 1221us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:27.758201 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:27.761135 (+ 2934us) server_negotiation.cc:407] Connection header received
0811 22:11:27.773679 (+ 12544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:27.773719 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:27.774605 (+ 886us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:27.775277 (+ 672us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:27.779122 (+ 3845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:27.780622 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:27.783595 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:27.784550 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:27.784785 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:27.793941 (+ 9156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:27.794004 (+ 63us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:27.794043 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:27.794093 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:27.798185 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:27.799326 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:27.799333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:27.799339 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:27.799558 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:27.800128 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:27.800134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:27.800137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:27.800833 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:27.801300 (+ 467us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:27.802329 (+ 1029us) server_negotiation.cc:299] Negotiation successful
0811 22:11:27.802781 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":800,"thread_start_us":426,"threads_started":1}
I20260811 22:11:27.807890 11058 sys_catalog.cc:564] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:11:27.808779 11066 raft_consensus.cc:686] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [term 1 LEADER]: Becoming Leader. State: Replica: c17e7d1b83384bc689066905288608fa, State: Running, Role: LEADER
I20260811 22:11:27.809921 11066 consensus_queue.cc:237] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17e7d1b83384bc689066905288608fa" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39285 } }
I20260811 22:11:27.815743 11070 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader c17e7d1b83384bc689066905288608fa. Latest consensus state: current_term: 1 leader_uuid: "c17e7d1b83384bc689066905288608fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17e7d1b83384bc689066905288608fa" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39285 } } }
I20260811 22:11:27.816285 11070 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:27.817131 11068 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c17e7d1b83384bc689066905288608fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17e7d1b83384bc689066905288608fa" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39285 } } }
I20260811 22:11:27.817565 11068 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:27.821965 11084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:11:27.830360 11084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:11:27.845665 11084 catalog_manager.cc:1347] Generated new cluster ID: e0747d2d724c4578807fdfe7c8dee121
I20260811 22:11:27.845806 11084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:11:27.862040 11084 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:11:27.863611 11084 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:11:27.876861 11084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c17e7d1b83384bc689066905288608fa: Generated new TSK 0
I20260811 22:11:27.877931 11084 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:11:27.935279 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:39285
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:28.319525 11098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:28.319820 11098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:28.319948 11098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:28.328542 11098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:28.328668 11098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:28.328799 11098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:11:28.340560 11098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:39285
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:28.341928 11098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:28.343482 11098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:28.359619 11105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:28.359575 11109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:28.359673 11106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:28.360555 11098 server_base.cc:1034] running on GCE node
I20260811 22:11:28.361426 11098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:28.362699 11098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:28.363926 11098 hybrid_clock.cc:648] HybridClock initialized: now 1786486288363879 us; error 34 us; skew 500 ppm
I20260811 22:11:28.364315 11098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:28 dist-test-slave-68xk krb5kdc[10945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486288, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:28.367923 11098 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:11:28.369722 11098 webserver.cc:460] Webserver started at http://127.10.173.129:40863/ using document root <none> and password file <none>
I20260811 22:11:28.370538 11098 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:28.370729 11098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:28.371143 11098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:28.374405 11098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "d5c1716ba5de423ead239411d9779348"
format_stamp: "Formatted at 2026-08-11 22:11:28 on dist-test-slave-68xk"
server_key: "888c0b55a109c957f30772980dd3ced1"
server_key_iv: "ab1568f3cbed809716576be2500e9828"
server_key_version: "encryptionkey@0"
I20260811 22:11:28.375341 11098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "d5c1716ba5de423ead239411d9779348"
format_stamp: "Formatted at 2026-08-11 22:11:28 on dist-test-slave-68xk"
server_key: "888c0b55a109c957f30772980dd3ced1"
server_key_iv: "ab1568f3cbed809716576be2500e9828"
server_key_version: "encryptionkey@0"
I20260811 22:11:28.381603 11098 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 22:11:28.385761 11120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:28.387257 11098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:11:28.387442 11098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "d5c1716ba5de423ead239411d9779348"
format_stamp: "Formatted at 2026-08-11 22:11:28 on dist-test-slave-68xk"
server_key: "888c0b55a109c957f30772980dd3ced1"
server_key_iv: "ab1568f3cbed809716576be2500e9828"
server_key_version: "encryptionkey@0"
I20260811 22:11:28.387697 11098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:28.401742 11098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:28.406325 11098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:28.406692 11098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:28.408371 11098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:28.410295 11098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:28.410411 11098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:28.410657 11098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:28.410751 11098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:28.464277 11098 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:32897
I20260811 22:11:28.464331 11275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:32897 every 8 connection(s)
I20260811 22:11:28.466200 11098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:11:28.476065 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11098
I20260811 22:11:28.476434 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:11:28.477258 10934 external_mini_cluster.cc:1428] Setting key a2a6217f8b23e37dd92d58b227f9e4fb
Aug 11 22:11:28 dist-test-slave-68xk krb5kdc[10945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486288, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:28.492697 11280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:28.470182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:41535 (local address 127.10.173.190:39285)
0811 22:11:28.470775 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:28.470779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:28.471682 (+ 903us) server_negotiation.cc:407] Connection header received
0811 22:11:28.473678 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:28.473684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:28.473801 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:28.473927 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:28.475568 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:28.476372 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:28.478186 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:28.478741 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:28.478865 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:28.482767 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:28.482785 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:28.482790 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:28.482869 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:28.488250 (+ 5381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:28.489222 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:28.489230 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:28.489236 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:28.489455 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:28.490004 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:28.490011 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:28.490016 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:28.490557 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:28.490736 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:28.491716 (+ 980us) server_negotiation.cc:299] Negotiation successful
0811 22:11:28.492035 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":360,"threads_started":1}
I20260811 22:11:28.493265 11282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:28.470767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:39285 (local address 127.10.173.129:41535)
0811 22:11:28.471523 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:28.471560 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:28.473405 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:28.474171 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:28.474180 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:28.474817 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:28.475391 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:28.475410 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:28.476549 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:28.476557 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:28.477966 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:28.477987 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:28.479018 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:28.479025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:28.479336 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:28.480069 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:28.480092 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:28.482356 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:28.488425 (+ 6069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:28.488436 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:28.488456 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:28.489068 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:28.489594 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:28.489601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:28.489607 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:28.489844 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:28.490719 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:28.490730 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:28.491188 (+ 458us) client_negotiation.cc:769] Sending connection context
0811 22:11:28.491477 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 22:11:28.491948 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":316,"threads_started":1}
I20260811 22:11:28.495328 11276 heartbeater.cc:347] Connected to a master server at 127.10.173.190:39285
I20260811 22:11:28.495854 11276 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:28.496948 11276 heartbeater.cc:511] Master 127.10.173.190:39285 requested a full tablet report, sending...
I20260811 22:11:28.501487 11007 ts_manager.cc:194] Registered new tserver with Master: d5c1716ba5de423ead239411d9779348 (127.10.173.129:32897)
I20260811 22:11:28.504715 11007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:41535
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:11:28.537918 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:39285
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:28.911646 11285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:28.911981 11285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:28.912135 11285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:28.919915 11285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:28.920032 11285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:28.920264 11285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:11:28.931298 11285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:39285
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:28.932827 11285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:28.934266 11285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:28.948999 11293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:28.949107 11296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:28.948944 11294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:28.950310 11285 server_base.cc:1034] running on GCE node
I20260811 22:11:28.951016 11285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:28.952039 11285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:28.953284 11285 hybrid_clock.cc:648] HybridClock initialized: now 1786486288953237 us; error 74 us; skew 500 ppm
I20260811 22:11:28.953627 11285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:28 dist-test-slave-68xk krb5kdc[10945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486288, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:28.957433 11285 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:11:28.959407 11285 webserver.cc:460] Webserver started at http://127.10.173.130:42809/ using document root <none> and password file <none>
I20260811 22:11:28.960143 11285 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:28.960275 11285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:28.960561 11285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:28.963798 11285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "202d48dd25bc4c05918d951588182038"
format_stamp: "Formatted at 2026-08-11 22:11:28 on dist-test-slave-68xk"
server_key: "2253ae7b1991e2e4be0fce1b673a9c71"
server_key_iv: "5ea9002c5c7f89d64b36bdedb224e23f"
server_key_version: "encryptionkey@0"
I20260811 22:11:28.964586 11285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "202d48dd25bc4c05918d951588182038"
format_stamp: "Formatted at 2026-08-11 22:11:28 on dist-test-slave-68xk"
server_key: "2253ae7b1991e2e4be0fce1b673a9c71"
server_key_iv: "5ea9002c5c7f89d64b36bdedb224e23f"
server_key_version: "encryptionkey@0"
I20260811 22:11:28.970149 11285 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:11:28.974113 11305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:28.975535 11285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:11:28.975708 11285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "202d48dd25bc4c05918d951588182038"
format_stamp: "Formatted at 2026-08-11 22:11:28 on dist-test-slave-68xk"
server_key: "2253ae7b1991e2e4be0fce1b673a9c71"
server_key_iv: "5ea9002c5c7f89d64b36bdedb224e23f"
server_key_version: "encryptionkey@0"
I20260811 22:11:28.975919 11285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:29.001902 11285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:29.007167 11285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:29.007495 11285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:29.010491 11285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:29.012663 11285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:29.012809 11285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 22:11:29.012948 11285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:29.013057 11285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:29.064551 11285 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:33573
I20260811 22:11:29.064646 11492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:33573 every 8 connection(s)
I20260811 22:11:29.065892 11285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:11:29.071823 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11285
I20260811 22:11:29.072039 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:11:29.072657 10934 external_mini_cluster.cc:1428] Setting key 0879845133bbc8ce9425e4314d10b65b
Aug 11 22:11:29 dist-test-slave-68xk krb5kdc[10945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486288, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:29.094856 11495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:29.068758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:58583 (local address 127.10.173.190:39285)
0811 22:11:29.069225 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:29.069232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:29.070291 (+ 1059us) server_negotiation.cc:407] Connection header received
0811 22:11:29.072751 (+ 2460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:29.072759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:29.072847 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:29.073008 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:29.075534 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.076323 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.080155 (+ 3832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.080735 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.080848 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:29.086523 (+ 5675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:29.086529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:29.086532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:29.086568 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:29.090914 (+ 4346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:29.091822 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:29.091831 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:29.091839 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:29.092044 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:29.092520 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:29.092529 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:29.092535 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:29.092997 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:29.093168 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:29.093940 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 22:11:29.094260 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":215,"threads_started":1}
I20260811 22:11:29.095537 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:29.069408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:39285 (local address 127.10.173.130:58583)
0811 22:11:29.070126 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:29.070163 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:29.072401 (+ 2238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:29.073307 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:29.073318 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:29.074369 (+ 1051us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:29.075313 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:29.075343 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.076519 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.076529 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:29.079881 (+ 3352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:29.079900 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.081060 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.081070 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:29.081545 (+ 475us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:29.082553 (+ 1008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:29.082591 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:29.086210 (+ 3619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:29.091125 (+ 4915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:29.091132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:29.091146 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:29.091632 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:29.092186 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:29.092189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:29.092192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:29.092387 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:29.093162 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:29.093168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:29.093510 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 22:11:29.093761 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 22:11:29.094234 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":286,"threads_started":1}
I20260811 22:11:29.097276 11493 heartbeater.cc:347] Connected to a master server at 127.10.173.190:39285
I20260811 22:11:29.097751 11493 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:29.098835 11493 heartbeater.cc:511] Master 127.10.173.190:39285 requested a full tablet report, sending...
I20260811 22:11:29.101017 11007 ts_manager.cc:194] Registered new tserver with Master: 202d48dd25bc4c05918d951588182038 (127.10.173.130:33573)
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
I20260811 22:11:29.102145 11007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:58583
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:11:29.133383 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:39285
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:46185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:11:29.509977 11276 heartbeater.cc:503] Master 127.10.173.190:39285 was elected leader, sending a full tablet report...
W20260811 22:11:29.554927 11502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:29.555358 11502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:29.555641 11502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:29.563985 11502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:29.564182 11502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:29.564451 11502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:11:29.575508 11502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:46185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:39285
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:29.576999 11502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:29.578541 11502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:29.593854 11508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:29.594235 11511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:29.594676 11509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:29.600461 11502 server_base.cc:1034] running on GCE node
I20260811 22:11:29.601147 11502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:29.602224 11502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:29.603475 11502 hybrid_clock.cc:648] HybridClock initialized: now 1786486289603415 us; error 56 us; skew 500 ppm
I20260811 22:11:29.603907 11502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:29 dist-test-slave-68xk krb5kdc[10945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486289, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:29.608043 11502 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:11:29.609690 11502 webserver.cc:460] Webserver started at http://127.10.173.131:45279/ using document root <none> and password file <none>
I20260811 22:11:29.610378 11502 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:29.610555 11502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:29.610848 11502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:29.614027 11502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "012f8690bfb1400d81a4fdac8df98404"
format_stamp: "Formatted at 2026-08-11 22:11:29 on dist-test-slave-68xk"
server_key: "a5eb4fa8369716a2c92ffb53e6944d93"
server_key_iv: "d0a1481611aed5bf09d07631f9eca482"
server_key_version: "encryptionkey@0"
I20260811 22:11:29.614899 11502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "012f8690bfb1400d81a4fdac8df98404"
format_stamp: "Formatted at 2026-08-11 22:11:29 on dist-test-slave-68xk"
server_key: "a5eb4fa8369716a2c92ffb53e6944d93"
server_key_iv: "d0a1481611aed5bf09d07631f9eca482"
server_key_version: "encryptionkey@0"
I20260811 22:11:29.620155 11502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 22:11:29.623692 11522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:29.625003 11502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:11:29.625196 11502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "012f8690bfb1400d81a4fdac8df98404"
format_stamp: "Formatted at 2026-08-11 22:11:29 on dist-test-slave-68xk"
server_key: "a5eb4fa8369716a2c92ffb53e6944d93"
server_key_iv: "d0a1481611aed5bf09d07631f9eca482"
server_key_version: "encryptionkey@0"
I20260811 22:11:29.625429 11502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:29.645404 11502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:29.650213 11502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:29.650735 11502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:29.651888 11502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:29.653609 11502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:29.653693 11502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:29.653883 11502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:29.653939 11502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:29.700894 11502 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:46649
I20260811 22:11:29.700925 11698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:46649 every 8 connection(s)
I20260811 22:11:29.702626 11502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:11:29.704387 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11502
I20260811 22:11:29.704761 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:11:29.705369 10934 external_mini_cluster.cc:1428] Setting key 8fc165821cbd3c88e305d179ccbe67b9
Aug 11 22:11:29 dist-test-slave-68xk krb5kdc[10945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486289, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:29.726086 11702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:29.705811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:53833 (local address 127.10.173.190:39285)
0811 22:11:29.707359 (+ 1548us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:29.707368 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:29.708062 (+ 694us) server_negotiation.cc:407] Connection header received
0811 22:11:29.710850 (+ 2788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:29.710854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:29.710919 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:29.711011 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:29.712704 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.713512 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.715464 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.715902 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.716002 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:29.719828 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:29.719833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:29.719836 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:29.719870 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:29.722964 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:29.723685 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:29.723691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:29.723695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:29.723832 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:29.724296 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:29.724300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:29.724302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:29.724557 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:29.724724 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:29.725471 (+ 747us) server_negotiation.cc:299] Negotiation successful
0811 22:11:29.725690 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1385,"thread_start_us":1249,"threads_started":1}
I20260811 22:11:29.727128 11703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:29.706239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:39285 (local address 127.10.173.131:53833)
0811 22:11:29.707859 (+ 1620us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:29.707922 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:29.710526 (+ 2604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:29.711343 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:29.711352 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:29.711933 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:29.712545 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:29.712564 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.713671 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.713677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:29.715337 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:29.715351 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.716111 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.716116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:29.716376 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:29.717083 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:29.717114 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:29.719616 (+ 2502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:29.723102 (+ 3486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:29.723111 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:29.723128 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:29.723553 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:29.723949 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:29.723954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:29.723958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:29.724158 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:29.724692 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:29.724700 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:29.725052 (+ 352us) client_negotiation.cc:769] Sending connection context
0811 22:11:29.725380 (+ 328us) client_negotiation.cc:240] Negotiation successful
0811 22:11:29.725930 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1257,"thread_start_us":1033,"threads_started":1}
I20260811 22:11:29.728574 11699 heartbeater.cc:347] Connected to a master server at 127.10.173.190:39285
I20260811 22:11:29.728941 11699 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:29.729840 11699 heartbeater.cc:511] Master 127.10.173.190:39285 requested a full tablet report, sending...
I20260811 22:11:29.731500 11007 ts_manager.cc:194] Registered new tserver with Master: 012f8690bfb1400d81a4fdac8df98404 (127.10.173.131:46649)
I20260811 22:11:29.732362 11007 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:53833
I20260811 22:11:29.735898 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:11:29 dist-test-slave-68xk krb5kdc[10945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486289, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:11:29 dist-test-slave-68xk krb5kdc[10945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486289, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:11:29.798146 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:29.780894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33354 (local address 127.10.173.129:32897)
0811 22:11:29.781589 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:29.781596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:29.781624 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:11:29.781687 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:29.781693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:29.781916 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:29.782049 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:29.783965 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.785016 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.786656 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.787238 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.787365 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:29.791711 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:29.791724 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:29.791765 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:29.791817 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:29.795135 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:29.795862 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:29.795870 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:29.795873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:29.796000 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:29.796428 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:29.796438 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:29.796441 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:29.796872 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:29.797193 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:29.797510 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 22:11:29.797789 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":361,"threads_started":1}
W20260811 22:11:29.799728 11156 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33354
Aug 11 22:11:29 dist-test-slave-68xk krb5kdc[10945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:11:29 dist-test-slave-68xk krb5kdc[10945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:11:29.853835 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:29.838211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33368 (local address 127.10.173.129:32897)
0811 22:11:29.838546 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:29.838552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:29.838818 (+ 266us) server_negotiation.cc:407] Connection header received
0811 22:11:29.838962 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:29.838966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:29.839056 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:29.839161 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:29.840704 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.841424 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.842973 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.843419 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.843522 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:29.847909 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:29.847916 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:29.847919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:29.847961 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:29.851034 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:29.851822 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:29.851827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:29.851830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:29.851983 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:29.852444 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:29.852450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:29.852452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:29.852775 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:29.852980 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:29.853338 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 22:11:29.853547 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260811 22:11:29.856052 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11098
I20260811 22:11:29.870576 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:29.857354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.10.173.129:32897)
0811 22:11:29.857578 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:29.857587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:29.858071 (+ 484us) server_negotiation.cc:407] Connection header received
0811 22:11:29.858163 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:29.858166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:29.858226 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:29.858325 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:29.859779 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.860484 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.861919 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.862535 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.862645 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:29.864880 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:29.864885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:29.864889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:29.864918 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:29.867767 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:29.868498 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:29.868502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:29.868505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:29.868647 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:29.869066 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:29.869070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:29.869072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:29.869441 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:29.869624 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:29.869956 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 22:11:29.870107 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:11:29.871379 11269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:29.966112 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11098
I20260811 22:11:29.990149 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11285
Aug 11 22:11:29 dist-test-slave-68xk krb5kdc[10945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:11:30.009538 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:29.991283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60878 (local address 127.10.173.130:33573)
0811 22:11:29.991905 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:29.991912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:29.991940 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:11:29.992005 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:29.992012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:29.992226 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:29.992357 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:29.993577 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.994941 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.996324 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:29.997124 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:29.997241 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:30.002117 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:30.002130 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:30.002147 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:30.002197 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:30.005619 (+ 3422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:30.006778 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:30.006787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:30.006790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:30.006957 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:30.007515 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:30.007519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:30.007521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:30.007932 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:30.008227 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:30.008869 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 22:11:30.009136 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":315,"threads_started":1}
I20260811 22:11:30.011178 11483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:30.098294 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11285
I20260811 22:11:30.127303 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11502
Aug 11 22:11:30 dist-test-slave-68xk krb5kdc[10945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:11:30.145232 11725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:30.128523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.10.173.131:46649)
0811 22:11:30.129165 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:30.129173 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:30.129205 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:11:30.129317 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:30.129324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:30.129536 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:30.129669 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:30.131232 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:30.132228 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:30.133695 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:30.134232 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:30.134338 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:30.138966 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:30.138977 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:30.138992 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:30.139029 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:30.142001 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:30.142844 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:30.142852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:30.142855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:30.142999 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:30.143458 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:30.143463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:30.143465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:30.143872 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:30.144180 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:30.144563 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 22:11:30.144817 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":370,"threads_started":1}
I20260811 22:11:30.146835 11693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:30.232378 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11502
I20260811 22:11:30.256893 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 10965
I20260811 22:11:30.258172 11050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:30.366657 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 10965
2026-08-11T22:11:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5037 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:11:30 dist-test-slave-68xk krb5kdc[11731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:11:30 dist-test-slave-68xk krb5kdc[11731](info): set up 2 sockets
Aug 11 22:11:30 dist-test-slave-68xk krb5kdc[11731](info): commencing operation
krb5kdc: starting...
W20260811 22:11:31.669469 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.007s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:11:31 dist-test-slave-68xk krb5kdc[11731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:11:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:11:31.867019 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:36647
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:36647
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:32.196122 11753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:32.196426 11753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:32.196517 11753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:32.204633 11753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:11:32.204746 11753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:32.204797 11753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:32.204855 11753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:11:32.204901 11753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:11:32.215049 11753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:36647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:36647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:32.216303 11753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:32.217581 11753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:32.229609 11758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:32.229660 11763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:32.229547 11759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:32.230593 11753 server_base.cc:1034] running on GCE node
I20260811 22:11:32.231371 11753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:32.232802 11753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:32.234051 11753 hybrid_clock.cc:648] HybridClock initialized: now 1786486292234002 us; error 55 us; skew 500 ppm
I20260811 22:11:32.234457 11753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:32 dist-test-slave-68xk krb5kdc[11731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486292, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:32.237660 11753 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:11:32.239174 11753 webserver.cc:460] Webserver started at http://127.10.173.190:46721/ using document root <none> and password file <none>
I20260811 22:11:32.239814 11753 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:32.239914 11753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:32.240183 11753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:32.243134 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "ce42ae6932374db1afed1edd714dea8f"
format_stamp: "Formatted at 2026-08-11 22:11:32 on dist-test-slave-68xk"
server_key: "49a95c180df14d77e85861a95b4dbf8d"
server_key_iv: "01a6e2ab86cf3ed30f24d86a518ca69b"
server_key_version: "encryptionkey@0"
I20260811 22:11:32.243834 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "ce42ae6932374db1afed1edd714dea8f"
format_stamp: "Formatted at 2026-08-11 22:11:32 on dist-test-slave-68xk"
server_key: "49a95c180df14d77e85861a95b4dbf8d"
server_key_iv: "01a6e2ab86cf3ed30f24d86a518ca69b"
server_key_version: "encryptionkey@0"
I20260811 22:11:32.249028 11753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:11:32.252830 11772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:32.254519 11753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:11:32.254726 11753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "ce42ae6932374db1afed1edd714dea8f"
format_stamp: "Formatted at 2026-08-11 22:11:32 on dist-test-slave-68xk"
server_key: "49a95c180df14d77e85861a95b4dbf8d"
server_key_iv: "01a6e2ab86cf3ed30f24d86a518ca69b"
server_key_version: "encryptionkey@0"
I20260811 22:11:32.254957 11753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:32.299711 11753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:32.304631 11753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:32.304962 11753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:32.325143 11753 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:36647
I20260811 22:11:32.325146 11860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:36647 every 8 connection(s)
I20260811 22:11:32.326629 11753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:11:32.331516 11861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:32.336593 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11753
I20260811 22:11:32.336851 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:11:32.337646 10934 external_mini_cluster.cc:1428] Setting key 6383763227db675dc2724b83716795a7
I20260811 22:11:32.343384 11861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f: Bootstrap starting.
I20260811 22:11:32.347214 11861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:32.348477 11861 log.cc:824] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:32.351603 11861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f: No bootstrap required, opened a new log
Aug 11 22:11:32 dist-test-slave-68xk krb5kdc[11731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:32.360447 11861 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce42ae6932374db1afed1edd714dea8f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36647 } }
I20260811 22:11:32.360860 11861 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:32.360999 11861 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce42ae6932374db1afed1edd714dea8f, State: Initialized, Role: FOLLOWER
I20260811 22:11:32.361709 11861 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce42ae6932374db1afed1edd714dea8f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36647 } }
I20260811 22:11:32.362012 11861 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:11:32.362172 11861 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:11:32.362370 11861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:32.362663 11865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:32.340094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53094 (local address 127.10.173.190:36647)
0811 22:11:32.341080 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:32.341096 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:32.341148 (+ 52us) server_negotiation.cc:407] Connection header received
0811 22:11:32.343156 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:32.343180 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:32.343654 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:32.343999 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:32.346101 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:32.347592 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:32.349419 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:32.350105 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:32.350329 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:32.354727 (+ 4398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:32.354737 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:32.354763 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:32.354800 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:32.358022 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:32.358850 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:32.358857 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:32.358864 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:32.359039 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:32.359444 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:32.359451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:32.359454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:32.359999 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:32.360304 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:32.360852 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 22:11:32.361344 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":464,"threads_started":1}
I20260811 22:11:32.365492 11861 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce42ae6932374db1afed1edd714dea8f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36647 } }
I20260811 22:11:32.366039 11861 leader_election.cc:302] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce42ae6932374db1afed1edd714dea8f; no voters:
I20260811 22:11:32.366698 11861 leader_election.cc:288] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:11:32.366824 11867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:32.367642 11867 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [term 1 LEADER]: Becoming Leader. State: Replica: ce42ae6932374db1afed1edd714dea8f, State: Running, Role: LEADER
I20260811 22:11:32.368216 11867 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce42ae6932374db1afed1edd714dea8f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36647 } }
I20260811 22:11:32.368911 11861 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:11:32.371248 11870 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce42ae6932374db1afed1edd714dea8f. Latest consensus state: current_term: 1 leader_uuid: "ce42ae6932374db1afed1edd714dea8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce42ae6932374db1afed1edd714dea8f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36647 } } }
I20260811 22:11:32.371701 11870 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:32.372462 11868 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce42ae6932374db1afed1edd714dea8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce42ae6932374db1afed1edd714dea8f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36647 } } }
I20260811 22:11:32.372810 11868 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:32.373080 11876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:11:32.378621 11876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:11:32.387553 11876 catalog_manager.cc:1347] Generated new cluster ID: d087ec6c10a54b1baf2bf4099ec7797b
I20260811 22:11:32.387687 11876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:11:32.401597 11876 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:11:32.403537 11876 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:11:32.420223 11876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce42ae6932374db1afed1edd714dea8f: Generated new TSK 0
I20260811 22:11:32.421330 11876 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:11:32.474460 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36647
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:32.809576 11897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:32.809926 11897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:32.810177 11897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:32.817510 11897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:32.817663 11897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:32.817852 11897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:11:32.827651 11897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:32.829057 11897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:32.830461 11897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:32.844501 11903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:32.844601 11904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:32.844575 11907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:32.848071 11897 server_base.cc:1034] running on GCE node
I20260811 22:11:32.848870 11897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:32.849917 11897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:32.851194 11897 hybrid_clock.cc:648] HybridClock initialized: now 1786486292851141 us; error 41 us; skew 500 ppm
I20260811 22:11:32.851586 11897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:32 dist-test-slave-68xk krb5kdc[11731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486292, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:32.855309 11897 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:11:32.857113 11897 webserver.cc:460] Webserver started at http://127.10.173.129:44967/ using document root <none> and password file <none>
I20260811 22:11:32.857923 11897 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:32.858085 11897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:32.858515 11897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:32.861608 11897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "ba2a15eea2d3438bae4f02c12f100a97"
format_stamp: "Formatted at 2026-08-11 22:11:32 on dist-test-slave-68xk"
server_key: "1f72cd8c64512b191d6a76b102be28fb"
server_key_iv: "4df3c2b540e737933a0adeef0c562d13"
server_key_version: "encryptionkey@0"
I20260811 22:11:32.862502 11897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "ba2a15eea2d3438bae4f02c12f100a97"
format_stamp: "Formatted at 2026-08-11 22:11:32 on dist-test-slave-68xk"
server_key: "1f72cd8c64512b191d6a76b102be28fb"
server_key_iv: "4df3c2b540e737933a0adeef0c562d13"
server_key_version: "encryptionkey@0"
I20260811 22:11:32.867889 11897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 22:11:32.871814 11916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:32.873287 11897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:11:32.873486 11897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "ba2a15eea2d3438bae4f02c12f100a97"
format_stamp: "Formatted at 2026-08-11 22:11:32 on dist-test-slave-68xk"
server_key: "1f72cd8c64512b191d6a76b102be28fb"
server_key_iv: "4df3c2b540e737933a0adeef0c562d13"
server_key_version: "encryptionkey@0"
I20260811 22:11:32.873697 11897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:32.891533 11897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:32.895985 11897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:32.896306 11897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:32.897393 11897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:32.898963 11897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:32.899075 11897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:32.899240 11897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:32.899338 11897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:32.936642 11897 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:39463
I20260811 22:11:32.936694 12091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:39463 every 8 connection(s)
I20260811 22:11:32.938036 11897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:11:32.947201 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11897
I20260811 22:11:32.947446 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:11:32.948298 10934 external_mini_cluster.cc:1428] Setting key 3558e7a64e7b013337405c9b289402d1
Aug 11 22:11:32 dist-test-slave-68xk krb5kdc[11731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486292, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:32.960922 12095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:32.940866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:55387 (local address 127.10.173.190:36647)
0811 22:11:32.941432 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:32.941440 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:32.942434 (+ 994us) server_negotiation.cc:407] Connection header received
0811 22:11:32.944311 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:32.944316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:32.944390 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:32.944496 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:32.945993 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:32.946881 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:32.949253 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:32.949840 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:32.949975 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:32.954759 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:32.954765 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:32.954768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:32.954803 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:32.957513 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:32.958426 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:32.958434 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:32.958438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:32.958585 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:32.959071 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:32.959079 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:32.959084 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:32.959395 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:32.959561 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:32.960306 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 22:11:32.960537 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":285,"threads_started":1}
I20260811 22:11:32.961939 12096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:32.941408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36647 (local address 127.10.173.129:55387)
0811 22:11:32.942232 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:32.942275 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:32.944047 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:32.944712 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:32.944720 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:32.945282 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:32.945831 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:32.945847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:32.947063 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:32.947069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:32.949082 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:32.949094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:32.950075 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:32.950083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:32.950398 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:32.951254 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:32.951286 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:32.954471 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:32.957700 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:32.957710 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:32.957728 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:32.958264 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:32.958713 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:32.958719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:32.958722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:32.958931 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:32.959534 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:32.959542 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:32.959912 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 22:11:32.960211 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 22:11:32.960767 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":319,"threads_started":1}
I20260811 22:11:32.963713 12093 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36647
I20260811 22:11:32.964036 12093 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:32.964792 12093 heartbeater.cc:511] Master 127.10.173.190:36647 requested a full tablet report, sending...
I20260811 22:11:32.967332 11803 ts_manager.cc:194] Registered new tserver with Master: ba2a15eea2d3438bae4f02c12f100a97 (127.10.173.129:39463)
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
I20260811 22:11:32.969266 11803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:55387
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:11:32.999861 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36647
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:33.339663 12099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:33.339985 12099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:33.340161 12099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:33.347437 12099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:33.347594 12099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:33.347774 12099 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:11:33.357690 12099 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:33.359192 12099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:33.360716 12099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:33.374226 12108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:33.374282 12113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:33.374497 12109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:33.375648 12099 server_base.cc:1034] running on GCE node
I20260811 22:11:33.376281 12099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:33.377269 12099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:33.378535 12099 hybrid_clock.cc:648] HybridClock initialized: now 1786486293378493 us; error 37 us; skew 500 ppm
I20260811 22:11:33.378855 12099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:33 dist-test-slave-68xk krb5kdc[11731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:33.382016 12099 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:11:33.383801 12099 webserver.cc:460] Webserver started at http://127.10.173.130:44613/ using document root <none> and password file <none>
I20260811 22:11:33.384508 12099 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:33.384671 12099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:33.384999 12099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:33.388072 12099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "3877c1edbc7b4b7295380ab782ea529e"
format_stamp: "Formatted at 2026-08-11 22:11:33 on dist-test-slave-68xk"
server_key: "97a306679fd31297fe90cf784f243596"
server_key_iv: "10398f4588109dd4664c67e427bfe0b8"
server_key_version: "encryptionkey@0"
I20260811 22:11:33.388907 12099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "3877c1edbc7b4b7295380ab782ea529e"
format_stamp: "Formatted at 2026-08-11 22:11:33 on dist-test-slave-68xk"
server_key: "97a306679fd31297fe90cf784f243596"
server_key_iv: "10398f4588109dd4664c67e427bfe0b8"
server_key_version: "encryptionkey@0"
I20260811 22:11:33.394312 12099 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:11:33.398064 12126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:33.399681 12099 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:11:33.399896 12099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "3877c1edbc7b4b7295380ab782ea529e"
format_stamp: "Formatted at 2026-08-11 22:11:33 on dist-test-slave-68xk"
server_key: "97a306679fd31297fe90cf784f243596"
server_key_iv: "10398f4588109dd4664c67e427bfe0b8"
server_key_version: "encryptionkey@0"
I20260811 22:11:33.400158 12099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:33.418119 12099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:33.423032 12099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:33.423360 12099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:33.424454 12099 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:33.426005 12099 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:33.426118 12099 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:33.426283 12099 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:33.426362 12099 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:33.463137 12099 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:33993
I20260811 22:11:33.463161 12305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:33993 every 8 connection(s)
I20260811 22:11:33.464512 12099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:11:33.472016 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 12099
I20260811 22:11:33.472227 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:11:33.472795 10934 external_mini_cluster.cc:1428] Setting key bd892c4db5f938bdd4bae552650e1fbc
Aug 11 22:11:33 dist-test-slave-68xk krb5kdc[11731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:33.486387 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:33.467391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:59997 (local address 127.10.173.190:36647)
0811 22:11:33.467838 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:33.467843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:33.468836 (+ 993us) server_negotiation.cc:407] Connection header received
0811 22:11:33.470857 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:33.470862 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:33.470932 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:33.471031 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:33.472952 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:33.473866 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:33.476075 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:33.476512 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:33.476617 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:33.480516 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:33.480521 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:33.480525 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:33.480556 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:33.483287 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:33.484017 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:33.484023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:33.484031 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:33.484163 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:33.484616 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:33.484621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:33.484629 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:33.484890 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:33.485073 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:33.485771 (+ 698us) server_negotiation.cc:299] Negotiation successful
0811 22:11:33.485996 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":215,"threads_started":1}
I20260811 22:11:33.487437 12310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:33.467948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36647 (local address 127.10.173.130:59997)
0811 22:11:33.468664 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:33.468701 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:33.470607 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:33.471239 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:33.471248 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:33.471927 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:33.472727 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:33.472762 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:33.474029 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:33.474033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:33.475926 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:33.475944 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:33.476769 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:33.476779 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:33.477125 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:33.477939 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:33.477969 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:33.480288 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:33.483435 (+ 3147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:33.483443 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:33.483459 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:33.483883 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:33.484288 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:33.484293 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:33.484297 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:33.484487 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:33.485022 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:33.485052 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:33.485388 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 22:11:33.485673 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 22:11:33.486195 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":277,"threads_started":1}
I20260811 22:11:33.489056 12307 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36647
I20260811 22:11:33.489373 12307 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:33.490118 12307 heartbeater.cc:511] Master 127.10.173.190:36647 requested a full tablet report, sending...
I20260811 22:11:33.491683 11803 ts_manager.cc:194] Registered new tserver with Master: 3877c1edbc7b4b7295380ab782ea529e (127.10.173.130:33993)
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
I20260811 22:11:33.492525 11803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:59997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:11:33.521658 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36647
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:33.848276 12314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:33.848554 12314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:33.848729 12314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:33.855996 12314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:33.856127 12314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:33.856338 12314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:11:33.866287 12314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36647
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:33.867758 12314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:33.869762 12314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:33.884235 12322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:33.884279 12327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:33.884315 12324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:33.885244 12314 server_base.cc:1034] running on GCE node
I20260811 22:11:33.885890 12314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:33.887161 12314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:33.888409 12314 hybrid_clock.cc:648] HybridClock initialized: now 1786486293888339 us; error 60 us; skew 500 ppm
I20260811 22:11:33.888713 12314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:33 dist-test-slave-68xk krb5kdc[11731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:33.892069 12314 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:11:33.893577 12314 webserver.cc:460] Webserver started at http://127.10.173.131:36767/ using document root <none> and password file <none>
I20260811 22:11:33.894248 12314 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:33.894356 12314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:33.894712 12314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:33.897810 12314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "9253cc997336482b94e04c8503d29d02"
format_stamp: "Formatted at 2026-08-11 22:11:33 on dist-test-slave-68xk"
server_key: "9e7fadc657df6fbc0a449e44865ae68b"
server_key_iv: "4daabd4e9ef582099a035c833eca1df4"
server_key_version: "encryptionkey@0"
I20260811 22:11:33.898705 12314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "9253cc997336482b94e04c8503d29d02"
format_stamp: "Formatted at 2026-08-11 22:11:33 on dist-test-slave-68xk"
server_key: "9e7fadc657df6fbc0a449e44865ae68b"
server_key_iv: "4daabd4e9ef582099a035c833eca1df4"
server_key_version: "encryptionkey@0"
I20260811 22:11:33.904253 12314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:11:33.908433 12335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:33.910020 12314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:11:33.910168 12314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "9253cc997336482b94e04c8503d29d02"
format_stamp: "Formatted at 2026-08-11 22:11:33 on dist-test-slave-68xk"
server_key: "9e7fadc657df6fbc0a449e44865ae68b"
server_key_iv: "4daabd4e9ef582099a035c833eca1df4"
server_key_version: "encryptionkey@0"
I20260811 22:11:33.910358 12314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:33.933250 12314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:33.939322 12314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:33.939703 12314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:33.941059 12314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:33.942936 12314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:33.943046 12314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:33.943194 12314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:33.943279 12314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:33.972714 12093 heartbeater.cc:503] Master 127.10.173.190:36647 was elected leader, sending a full tablet report...
I20260811 22:11:33.981019 12314 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:36419
I20260811 22:11:33.981106 12513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:36419 every 8 connection(s)
I20260811 22:11:33.982499 12314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:11:33.991345 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 12314
I20260811 22:11:33.991559 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:11:33.992291 10934 external_mini_cluster.cc:1428] Setting key b45587ec7df54596206eb46eac70cca1
Aug 11 22:11:33 dist-test-slave-68xk krb5kdc[11731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:34.006546 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:33.985703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:59761 (local address 127.10.173.190:36647)
0811 22:11:33.985999 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:33.986006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:33.987562 (+ 1556us) server_negotiation.cc:407] Connection header received
0811 22:11:33.989707 (+ 2145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:33.989711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:33.989800 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:33.989899 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:33.992288 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:33.993328 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:33.995025 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:33.995431 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:33.995583 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:33.999799 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:33.999805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:33.999809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:33.999848 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:34.002772 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.003616 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.003677 (+ 61us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.003682 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.003854 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.004411 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.004415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.004417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.004796 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:34.004956 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.005834 (+ 878us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.006079 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260811 22:11:34.007939 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:33.986266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36647 (local address 127.10.173.131:59761)
0811 22:11:33.987376 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:33.987432 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:33.989389 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:33.990236 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:33.990245 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:33.991063 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:33.992057 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:33.992088 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:33.993511 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:33.993516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:33.994861 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:33.994875 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:33.995792 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:33.995801 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:33.996036 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:33.996770 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:33.996798 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:33.999501 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:34.002922 (+ 3421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:34.002930 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:34.002944 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:34.003455 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:34.004018 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:34.004026 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:34.004032 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:34.004275 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:34.004947 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:34.004953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:34.005421 (+ 468us) client_negotiation.cc:769] Sending connection context
0811 22:11:34.005689 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 22:11:34.006289 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":410,"threads_started":1}
I20260811 22:11:34.009618 12514 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36647
I20260811 22:11:34.009907 12514 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:34.010737 12514 heartbeater.cc:511] Master 127.10.173.190:36647 requested a full tablet report, sending...
I20260811 22:11:34.012259 11803 ts_manager.cc:194] Registered new tserver with Master: 9253cc997336482b94e04c8503d29d02 (127.10.173.131:36419)
I20260811 22:11:34.013088 11803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:59761
I20260811 22:11:34.021009 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:11:34.043494 12309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.028708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53096 (local address 127.10.173.190:36647)
0811 22:11:34.028888 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.028893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.029582 (+ 689us) server_negotiation.cc:407] Connection header received
0811 22:11:34.029807 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.029813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.029898 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.030008 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:34.032018 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.032903 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.034336 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.034864 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.035012 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.037639 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:34.037644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:34.037647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:34.037678 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:34.040666 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.041338 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.041346 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.041349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.041522 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.041928 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.041935 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.041939 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.042247 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:34.042548 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.042933 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.043159 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:11:34.052314 11803 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:11:34.060935 11803 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:11:34.110174 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.079525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.10.173.129:39463)
0811 22:11:34.101334 (+ 21809us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.101342 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.101367 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:11:34.101446 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.101452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.101678 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.101899 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:34.102960 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.104996 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.107268 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.108782 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.108995 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.109207 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.109644 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.109828 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21630,"thread_start_us":21458,"threads_started":1}
I20260811 22:11:34.110267 12542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.078817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:39463 (local address 127.0.0.1:58856)
0811 22:11:34.089633 (+ 10816us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:34.089694 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:34.090032 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:34.101887 (+ 11855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:34.101893 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:34.101923 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:34.102725 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:34.102743 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.105189 (+ 2446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.105193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:34.107098 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:34.107111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.109055 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.109062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:34.109323 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.109392 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 22:11:34.109525 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 22:11:34.109758 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10555,"mutex_wait_us":4,"thread_start_us":17163,"threads_started":1}
I20260811 22:11:34.111799 12543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.080241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56462 (local address 127.10.173.130:33993)
0811 22:11:34.098961 (+ 18720us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.098969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.098996 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:11:34.099140 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.099146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.099357 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.101569 (+ 2212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:34.101950 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.103843 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.107644 (+ 3801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.109116 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.109286 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.109456 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.111134 (+ 1678us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.111303 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18575,"thread_start_us":18484,"threads_started":1}
I20260811 22:11:34.112381 12534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.078859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:36419 (local address 127.0.0.1:46888)
0811 22:11:34.095370 (+ 16511us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:34.095395 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:34.095563 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:34.103831 (+ 8268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:34.103836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:34.103871 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:34.104752 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:34.104769 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.107583 (+ 2814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.107588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:34.109336 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:34.109350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.111421 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.111428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:34.111610 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.111652 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 22:11:34.111788 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 22:11:34.111931 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16324,"thread_start_us":16829,"threads_started":1}
I20260811 22:11:34.112447 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.079283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46888 (local address 127.10.173.131:36419)
0811 22:11:34.103124 (+ 23841us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.103134 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.103171 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:11:34.103316 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.103325 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.103587 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.103804 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:34.105005 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.107297 (+ 2292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.109539 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.111188 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.111422 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.111595 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.111846 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.112025 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23568,"thread_start_us":23346,"threads_started":1}
I20260811 22:11:34.112514 12533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.079551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:33993 (local address 127.0.0.1:56462)
0811 22:11:34.095767 (+ 16216us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:34.095784 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:34.095900 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:34.099659 (+ 3759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:34.099667 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:34.099712 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:34.100419 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:34.100434 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.104068 (+ 3634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.104078 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:34.107436 (+ 3358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:34.107453 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.109389 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.109396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:34.110589 (+ 1193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.110622 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:11:34.110713 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:11:34.110845 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16094,"thread_start_us":9664,"threads_started":1}
I20260811 22:11:34.115792 11991 tablet_service.cc:1467] Processing CreateTablet for tablet d71f2ba26a4044b7b50cdf8aaa2bd43b (DEFAULT_TABLE table=test-table [id=58686745bf534196b44d0bc3583b3fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:34.116892 12415 tablet_service.cc:1467] Processing CreateTablet for tablet d71f2ba26a4044b7b50cdf8aaa2bd43b (DEFAULT_TABLE table=test-table [id=58686745bf534196b44d0bc3583b3fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:34.116956 12208 tablet_service.cc:1467] Processing CreateTablet for tablet d71f2ba26a4044b7b50cdf8aaa2bd43b (DEFAULT_TABLE table=test-table [id=58686745bf534196b44d0bc3583b3fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:34.117648 11991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71f2ba26a4044b7b50cdf8aaa2bd43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:34.118299 12415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71f2ba26a4044b7b50cdf8aaa2bd43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:34.118441 12208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d71f2ba26a4044b7b50cdf8aaa2bd43b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:34.131937 12545 tablet_bootstrap.cc:492] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02: Bootstrap starting.
I20260811 22:11:34.134128 12547 tablet_bootstrap.cc:492] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97: Bootstrap starting.
I20260811 22:11:34.136504 12548 tablet_bootstrap.cc:492] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e: Bootstrap starting.
I20260811 22:11:34.136679 12545 tablet_bootstrap.cc:654] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:34.138476 12547 tablet_bootstrap.cc:654] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:34.138525 12545 log.cc:824] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:34.139863 12548 tablet_bootstrap.cc:654] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:34.140096 12547 log.cc:824] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:34.141232 12548 log.cc:824] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:34.143762 12545 tablet_bootstrap.cc:492] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02: No bootstrap required, opened a new log
I20260811 22:11:34.144137 12545 ts_tablet_manager.cc:1397] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.005s
I20260811 22:11:34.144394 12547 tablet_bootstrap.cc:492] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97: No bootstrap required, opened a new log
I20260811 22:11:34.144718 12547 ts_tablet_manager.cc:1397] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 22:11:34.146970 12548 tablet_bootstrap.cc:492] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e: No bootstrap required, opened a new log
I20260811 22:11:34.147295 12548 ts_tablet_manager.cc:1397] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260811 22:11:34.153548 12545 raft_consensus.cc:348] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2a15eea2d3438bae4f02c12f100a97" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 39463 } } peers { permanent_uuid: "9253cc997336482b94e04c8503d29d02" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36419 } } peers { permanent_uuid: "3877c1edbc7b4b7295380ab782ea529e" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 33993 } }
I20260811 22:11:34.154031 12545 raft_consensus.cc:374] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:34.154201 12545 raft_consensus.cc:729] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9253cc997336482b94e04c8503d29d02, State: Initialized, Role: FOLLOWER
I20260811 22:11:34.154961 12545 consensus_queue.cc:260] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2a15eea2d3438bae4f02c12f100a97" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 39463 } } peers { permanent_uuid: "9253cc997336482b94e04c8503d29d02" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36419 } } peers { permanent_uuid: "3877c1edbc7b4b7295380ab782ea529e" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 33993 } }
I20260811 22:11:34.156455 12514 heartbeater.cc:503] Master 127.10.173.190:36647 was elected leader, sending a full tablet report...
I20260811 22:11:34.156502 12547 raft_consensus.cc:348] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2a15eea2d3438bae4f02c12f100a97" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 39463 } } peers { permanent_uuid: "9253cc997336482b94e04c8503d29d02" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36419 } } peers { permanent_uuid: "3877c1edbc7b4b7295380ab782ea529e" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 33993 } }
I20260811 22:11:34.156934 12547 raft_consensus.cc:374] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:34.157024 12547 raft_consensus.cc:729] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba2a15eea2d3438bae4f02c12f100a97, State: Initialized, Role: FOLLOWER
I20260811 22:11:34.157274 12545 ts_tablet_manager.cc:1428] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260811 22:11:34.157608 12547 consensus_queue.cc:260] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2a15eea2d3438bae4f02c12f100a97" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 39463 } } peers { permanent_uuid: "9253cc997336482b94e04c8503d29d02" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36419 } } peers { permanent_uuid: "3877c1edbc7b4b7295380ab782ea529e" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 33993 } }
I20260811 22:11:34.158885 12548 raft_consensus.cc:348] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2a15eea2d3438bae4f02c12f100a97" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 39463 } } peers { permanent_uuid: "9253cc997336482b94e04c8503d29d02" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36419 } } peers { permanent_uuid: "3877c1edbc7b4b7295380ab782ea529e" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 33993 } }
I20260811 22:11:34.159384 12548 raft_consensus.cc:374] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:34.159547 12548 raft_consensus.cc:729] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3877c1edbc7b4b7295380ab782ea529e, State: Initialized, Role: FOLLOWER
I20260811 22:11:34.159889 12547 ts_tablet_manager.cc:1428] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260811 22:11:34.160331 12548 consensus_queue.cc:260] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2a15eea2d3438bae4f02c12f100a97" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 39463 } } peers { permanent_uuid: "9253cc997336482b94e04c8503d29d02" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36419 } } peers { permanent_uuid: "3877c1edbc7b4b7295380ab782ea529e" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 33993 } }
I20260811 22:11:34.162463 12307 heartbeater.cc:503] Master 127.10.173.190:36647 was elected leader, sending a full tablet report...
I20260811 22:11:34.162896 12548 ts_tablet_manager.cc:1428] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
W20260811 22:11:34.190768 12094 tablet.cc:2369] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:11:34.201164 12553 raft_consensus.cc:482] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:34.201505 12553 raft_consensus.cc:504] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2a15eea2d3438bae4f02c12f100a97" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 39463 } } peers { permanent_uuid: "9253cc997336482b94e04c8503d29d02" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36419 } } peers { permanent_uuid: "3877c1edbc7b4b7295380ab782ea529e" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 33993 } }
I20260811 22:11:34.203716 12553 leader_election.cc:288] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9253cc997336482b94e04c8503d29d02 (127.10.173.131:36419), 3877c1edbc7b4b7295380ab782ea529e (127.10.173.130:33993)
I20260811 22:11:34.214095 12543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.204458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:51533 (local address 127.10.173.130:33993)
0811 22:11:34.204723 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.204732 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.205273 (+ 541us) server_negotiation.cc:407] Connection header received
0811 22:11:34.205522 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.205526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.205630 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.205743 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:34.206647 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.207823 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.211476 (+ 3653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.212736 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.212905 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.212979 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.213654 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.213844 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 22:11:34.214095 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.204209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:52627 (local address 127.10.173.131:36419)
0811 22:11:34.204488 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.204498 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.205602 (+ 1104us) server_negotiation.cc:407] Connection header received
0811 22:11:34.205748 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.205753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.205826 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.205915 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:34.206786 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.207875 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.211476 (+ 3601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.212739 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.212911 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.212979 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.213653 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.213805 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:11:34.214434 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.204148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:36419 (local address 127.10.173.129:52627)
0811 22:11:34.205436 (+ 1288us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:34.205451 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:34.205628 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:34.205950 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:34.205957 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:34.205997 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:34.206578 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:34.206592 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.208043 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.208047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:34.211275 (+ 3228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:34.211299 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.213133 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.213140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:34.213385 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.213454 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 22:11:34.213598 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 22:11:34.213769 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1169,"mutex_wait_us":264,"thread_start_us":557,"threads_started":1}
I20260811 22:11:34.214463 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.204104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:33993 (local address 127.10.173.129:51533)
0811 22:11:34.205098 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:34.205164 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:34.205359 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:34.205794 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:34.205797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:34.205832 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:34.206369 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:34.206394 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.208041 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.208047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:34.211298 (+ 3251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:34.211309 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.213122 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.213131 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:34.213370 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.213508 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 22:11:34.213598 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:11:34.213769 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":287,"thread_start_us":945,"threads_started":1}
I20260811 22:11:34.215758 12240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71f2ba26a4044b7b50cdf8aaa2bd43b" candidate_uuid: "ba2a15eea2d3438bae4f02c12f100a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3877c1edbc7b4b7295380ab782ea529e" is_pre_election: true
I20260811 22:11:34.215919 12449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71f2ba26a4044b7b50cdf8aaa2bd43b" candidate_uuid: "ba2a15eea2d3438bae4f02c12f100a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9253cc997336482b94e04c8503d29d02" is_pre_election: true
I20260811 22:11:34.216557 12240 raft_consensus.cc:2454] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2a15eea2d3438bae4f02c12f100a97 in term 0.
I20260811 22:11:34.216593 12449 raft_consensus.cc:2454] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba2a15eea2d3438bae4f02c12f100a97 in term 0.
W20260811 22:11:34.216763 12308 tablet.cc:2369] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:11:34.217494 11917 leader_election.cc:302] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3877c1edbc7b4b7295380ab782ea529e, ba2a15eea2d3438bae4f02c12f100a97; no voters:
I20260811 22:11:34.217941 12553 raft_consensus.cc:2781] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:34.218104 12553 raft_consensus.cc:482] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:34.218250 12553 raft_consensus.cc:3037] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:34.220695 12553 raft_consensus.cc:504] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2a15eea2d3438bae4f02c12f100a97" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 39463 } } peers { permanent_uuid: "9253cc997336482b94e04c8503d29d02" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36419 } } peers { permanent_uuid: "3877c1edbc7b4b7295380ab782ea529e" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 33993 } }
I20260811 22:11:34.221426 12553 leader_election.cc:288] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [CANDIDATE]: Term 1 election: Requested vote from peers 9253cc997336482b94e04c8503d29d02 (127.10.173.131:36419), 3877c1edbc7b4b7295380ab782ea529e (127.10.173.130:33993)
I20260811 22:11:34.221830 12449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71f2ba26a4044b7b50cdf8aaa2bd43b" candidate_uuid: "ba2a15eea2d3438bae4f02c12f100a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9253cc997336482b94e04c8503d29d02"
I20260811 22:11:34.221992 12240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d71f2ba26a4044b7b50cdf8aaa2bd43b" candidate_uuid: "ba2a15eea2d3438bae4f02c12f100a97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3877c1edbc7b4b7295380ab782ea529e"
I20260811 22:11:34.222056 12449 raft_consensus.cc:3037] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:34.222242 12240 raft_consensus.cc:3037] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:34.225673 12449 raft_consensus.cc:2454] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2a15eea2d3438bae4f02c12f100a97 in term 1.
I20260811 22:11:34.225728 12240 raft_consensus.cc:2454] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 3877c1edbc7b4b7295380ab782ea529e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba2a15eea2d3438bae4f02c12f100a97 in term 1.
I20260811 22:11:34.226140 11917 leader_election.cc:302] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3877c1edbc7b4b7295380ab782ea529e, ba2a15eea2d3438bae4f02c12f100a97; no voters:
I20260811 22:11:34.226462 12553 raft_consensus.cc:2781] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:34.226979 12553 raft_consensus.cc:686] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [term 1 LEADER]: Becoming Leader. State: Replica: ba2a15eea2d3438bae4f02c12f100a97, State: Running, Role: LEADER
I20260811 22:11:34.227543 12553 consensus_queue.cc:237] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2a15eea2d3438bae4f02c12f100a97" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 39463 } } peers { permanent_uuid: "9253cc997336482b94e04c8503d29d02" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36419 } } peers { permanent_uuid: "3877c1edbc7b4b7295380ab782ea529e" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 33993 } }
I20260811 22:11:34.233131 11803 catalog_manager.cc:5495] T d71f2ba26a4044b7b50cdf8aaa2bd43b P ba2a15eea2d3438bae4f02c12f100a97 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba2a15eea2d3438bae4f02c12f100a97 (127.10.173.129). New cstate: current_term: 1 leader_uuid: "ba2a15eea2d3438bae4f02c12f100a97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba2a15eea2d3438bae4f02c12f100a97" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 39463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9253cc997336482b94e04c8503d29d02" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3877c1edbc7b4b7295380ab782ea529e" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 33993 } health_report { overall_health: UNKNOWN } } }
W20260811 22:11:34.234272 12515 tablet.cc:2369] T d71f2ba26a4044b7b50cdf8aaa2bd43b P 9253cc997336482b94e04c8503d29d02: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:11:34 dist-test-slave-68xk krb5kdc[11731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:11:34.273340 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.257204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.10.173.129:39463)
0811 22:11:34.257490 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.257497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.258027 (+ 530us) server_negotiation.cc:407] Connection header received
0811 22:11:34.258260 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.258264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.258349 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.258524 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:34.260174 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.260956 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.262245 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.262712 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.262860 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.267250 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:34.267265 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:34.267283 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:34.267316 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:34.270372 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.271146 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.271162 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.271165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.271343 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.271795 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.271800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.271805 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.272149 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:34.272445 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.272775 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.273010 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
Aug 11 22:11:34 dist-test-slave-68xk krb5kdc[11731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486294, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:11:34 dist-test-slave-68xk krb5kdc[11731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486294, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:11:34.335326 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.319756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.10.173.129:39463)
0811 22:11:34.320005 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.320015 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.320413 (+ 398us) server_negotiation.cc:407] Connection header received
0811 22:11:34.320643 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.320647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.320715 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.320838 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:34.322502 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.323392 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.324740 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.325201 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.325348 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.329732 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:34.329737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:34.329740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:34.329781 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:34.332452 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.333095 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.333103 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.333106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.333250 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.333695 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.333700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.333703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.333987 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:34.334290 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.334651 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.334903 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 22:11:34 dist-test-slave-68xk krb5kdc[11731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:11:34 dist-test-slave-68xk krb5kdc[11731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:11:34.389909 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.374122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58880 (local address 127.10.173.129:39463)
0811 22:11:34.374408 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.374449 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.374761 (+ 312us) server_negotiation.cc:407] Connection header received
0811 22:11:34.374942 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.374946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.375028 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.375114 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:34.376697 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.377512 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.378993 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.379533 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.379632 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.384388 (+ 4756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:34.384394 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:34.384397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:34.384432 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:34.387548 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.388293 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.388297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.388299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.388421 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.388761 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.388764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.388766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.389018 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:34.389203 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.389470 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.389628 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260811 22:11:34.392330 11959 tablet_service.cc:2908] Rejecting scan request for tablet d71f2ba26a4044b7b50cdf8aaa2bd43b: Uninitialized: clean time has not yet been initialized
I20260811 22:11:34.396135 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11897
I20260811 22:11:34.410193 12541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.397243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.10.173.129:39463)
0811 22:11:34.397408 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.397413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.397878 (+ 465us) server_negotiation.cc:407] Connection header received
0811 22:11:34.398023 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.398026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.398084 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.398159 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:34.399435 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.400164 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.401482 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.401880 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.401971 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.404598 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:34.404603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:34.404606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:34.404637 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:34.407785 (+ 3148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.408504 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.408508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.408511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.408628 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.408994 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.408997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.408999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.409245 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:34.409427 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.409737 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.409932 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:11:34.410990 12087 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:34.504985 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11897
I20260811 22:11:34.533636 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12099
Aug 11 22:11:34 dist-test-slave-68xk krb5kdc[11731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:11:34.551234 12543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.534574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.10.173.130:33993)
0811 22:11:34.534802 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.534809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.534935 (+ 126us) server_negotiation.cc:407] Connection header received
0811 22:11:34.535170 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.535177 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.535272 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.535395 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:34.537434 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.538228 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.539710 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.540141 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.540241 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.544669 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:34.544684 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:34.544703 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:34.544742 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:34.548058 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.548800 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.548809 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.548812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.548966 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.549464 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.549468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.549470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.549897 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:34.550357 (+ 460us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.550627 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.550815 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:11:34.551920 12301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:34.644205 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 12099
I20260811 22:11:34.671160 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12314
Aug 11 22:11:34 dist-test-slave-68xk krb5kdc[11731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:11:34.687668 12537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:34.672019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46894 (local address 127.10.173.131:36419)
0811 22:11:34.672221 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:34.672226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:34.672247 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:11:34.672376 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:34.672379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:34.672450 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:34.672539 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:34.673857 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.674880 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.676358 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:34.676761 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:34.676869 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:34.681641 (+ 4772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:34.681657 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:34.681676 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:34.681714 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:34.684902 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.685630 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.685637 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.685639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.685782 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:34.686133 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:34.686136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:34.686138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:34.686642 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:34.686879 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:34.687215 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:11:34.687398 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:11:34.688416 12504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:34.777592 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 12314
I20260811 22:11:34.805401 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11753
I20260811 22:11:34.806782 11856 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:34.918219 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11753
2026-08-11T22:11:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4559 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:11:35 dist-test-slave-68xk krb5kdc[12590](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:11:35 dist-test-slave-68xk krb5kdc[12590](info): set up 2 sockets
Aug 11 22:11:35 dist-test-slave-68xk krb5kdc[12590](info): commencing operation
krb5kdc: starting...
W20260811 22:11:36.250162 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:11:36 dist-test-slave-68xk krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:11:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:11:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:11:36.462312 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:37173
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:37173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:36.819190 12613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:36.819748 12613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:36.819958 12613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:36.835350 12613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:11:36.835597 12613 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:11:36.835721 12613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:36.835863 12613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:36.835984 12613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:11:36.836134 12613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:11:36.856799 12613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:37173
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:37173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:36.858944 12613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:36.861043 12613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:36.879702 12623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:36.879665 12624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:36.880614 12626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:36.880695 12613 server_base.cc:1034] running on GCE node
I20260811 22:11:36.881644 12613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:36.883349 12613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:36.884624 12613 hybrid_clock.cc:648] HybridClock initialized: now 1786486296884535 us; error 67 us; skew 500 ppm
I20260811 22:11:36.885007 12613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:36 dist-test-slave-68xk krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:36.893810 12613 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:11:36.895711 12613 webserver.cc:460] Webserver started at http://127.10.173.190:37129/ using document root <none> and password file <none>
I20260811 22:11:36.896528 12613 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:36.896653 12613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:36.897043 12613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:36.900556 12613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "14c04cbbdcc24bd78b272cf3524edf76"
format_stamp: "Formatted at 2026-08-11 22:11:36 on dist-test-slave-68xk"
server_key: "7f1344f73d9b6cea29c305e3386b38ea"
server_key_iv: "0fadf57fdfe8585cb3fd36d055f1b569"
server_key_version: "encryptionkey@0"
I20260811 22:11:36.901475 12613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "14c04cbbdcc24bd78b272cf3524edf76"
format_stamp: "Formatted at 2026-08-11 22:11:36 on dist-test-slave-68xk"
server_key: "7f1344f73d9b6cea29c305e3386b38ea"
server_key_iv: "0fadf57fdfe8585cb3fd36d055f1b569"
server_key_version: "encryptionkey@0"
I20260811 22:11:36.907922 12613 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 22:11:36.912171 12635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:36.914227 12613 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:11:36.914511 12613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "14c04cbbdcc24bd78b272cf3524edf76"
format_stamp: "Formatted at 2026-08-11 22:11:36 on dist-test-slave-68xk"
server_key: "7f1344f73d9b6cea29c305e3386b38ea"
server_key_iv: "0fadf57fdfe8585cb3fd36d055f1b569"
server_key_version: "encryptionkey@0"
I20260811 22:11:36.914762 12613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:36.934506 12613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:36.939977 12613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:36.940292 12613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:36.965590 12613 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:37173
I20260811 22:11:36.965624 12723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:37173 every 8 connection(s)
I20260811 22:11:36.967154 12613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:11:36.971272 12724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:36.974037 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 12613
I20260811 22:11:36.974336 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:11:36.975437 10934 external_mini_cluster.cc:1428] Setting key 55396edd17b146c003e92fc9124112c0
I20260811 22:11:36.986280 12724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76: Bootstrap starting.
I20260811 22:11:36.989990 12724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:36.991431 12724 log.cc:824] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:36.995043 12724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76: No bootstrap required, opened a new log
Aug 11 22:11:36 dist-test-slave-68xk krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:37.005647 12724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c04cbbdcc24bd78b272cf3524edf76" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37173 } }
I20260811 22:11:37.006248 12724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:37.006386 12724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14c04cbbdcc24bd78b272cf3524edf76, State: Initialized, Role: FOLLOWER
I20260811 22:11:37.007215 12724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c04cbbdcc24bd78b272cf3524edf76" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37173 } }
I20260811 22:11:37.007660 12724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:11:37.007896 12724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:11:37.008173 12724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:37.011898 12724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c04cbbdcc24bd78b272cf3524edf76" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37173 } }
I20260811 22:11:37.012648 12724 leader_election.cc:302] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14c04cbbdcc24bd78b272cf3524edf76; no voters:
I20260811 22:11:37.012964 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:36.978019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37926 (local address 127.10.173.190:37173)
0811 22:11:36.984052 (+ 6033us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:36.984070 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:36.984136 (+ 66us) server_negotiation.cc:407] Connection header received
0811 22:11:36.986651 (+ 2515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:36.986683 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:36.987241 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:36.987705 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:36.990009 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:36.992194 (+ 2185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:36.993892 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:36.995413 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:36.995684 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:37.001731 (+ 6047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:37.001743 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:37.001771 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:37.001835 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:37.006628 (+ 4793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:37.007648 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:37.007659 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:37.007667 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:37.007894 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:37.008396 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:37.008404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:37.008409 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:37.009221 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:37.009523 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:37.010907 (+ 1384us) server_negotiation.cc:299] Negotiation successful
0811 22:11:37.011374 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5754,"thread_start_us":5864,"threads_started":1}
I20260811 22:11:37.013474 12724 leader_election.cc:288] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:11:37.013657 12734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:37.015972 12724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:11:37.017971 12734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [term 1 LEADER]: Becoming Leader. State: Replica: 14c04cbbdcc24bd78b272cf3524edf76, State: Running, Role: LEADER
I20260811 22:11:37.019835 12734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c04cbbdcc24bd78b272cf3524edf76" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37173 } }
I20260811 22:11:37.037608 12736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14c04cbbdcc24bd78b272cf3524edf76. Latest consensus state: current_term: 1 leader_uuid: "14c04cbbdcc24bd78b272cf3524edf76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c04cbbdcc24bd78b272cf3524edf76" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37173 } } }
I20260811 22:11:37.038127 12736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:37.038913 12735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14c04cbbdcc24bd78b272cf3524edf76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c04cbbdcc24bd78b272cf3524edf76" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37173 } } }
I20260811 22:11:37.039255 12735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76 [sys.catalog]: This master's current role is: LEADER
W20260811 22:11:37.054054 12762 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:11:37.054206 12762 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:11:37.054319 12763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:11:37.056504 12763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:11:37.065553 12763 catalog_manager.cc:1347] Generated new cluster ID: da814095c31a43788fcff799aa32c3cd
I20260811 22:11:37.065667 12763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:11:37.077845 12763 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:11:37.079806 12763 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:11:37.096779 12763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76: Generated new TSK 0
I20260811 22:11:37.097900 12763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:11:37.159534 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37173
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:37.504305 12775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:37.504635 12775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:37.504806 12775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:37.512631 12775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:37.512768 12775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:37.512905 12775 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:11:37.513034 12775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:11:37.523425 12775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.173.190:37173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:37.524775 12775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:37.526114 12775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:37.539362 12788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:37.539433 12793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:37.539443 12789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:37.540637 12775 server_base.cc:1034] running on GCE node
I20260811 22:11:37.541226 12775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:37.542091 12775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:37.543305 12775 hybrid_clock.cc:648] HybridClock initialized: now 1786486297543246 us; error 36 us; skew 500 ppm
I20260811 22:11:37.543610 12775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:37 dist-test-slave-68xk krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486297, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:37.546828 12775 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:11:37.548223 12775 webserver.cc:460] Webserver started at http://127.10.173.129:39517/ using document root <none> and password file <none>
I20260811 22:11:37.548897 12775 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:37.548983 12775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:37.549247 12775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:37.552518 12775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "4d8a01071ab348fda32ace0e88684204"
format_stamp: "Formatted at 2026-08-11 22:11:37 on dist-test-slave-68xk"
server_key: "7bbe9cd3ccdd27c06a347ca64e34cdce"
server_key_iv: "ee4f1e91b947b769a611a8d5ce9a4ee1"
server_key_version: "encryptionkey@0"
I20260811 22:11:37.553282 12775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "4d8a01071ab348fda32ace0e88684204"
format_stamp: "Formatted at 2026-08-11 22:11:37 on dist-test-slave-68xk"
server_key: "7bbe9cd3ccdd27c06a347ca64e34cdce"
server_key_iv: "ee4f1e91b947b769a611a8d5ce9a4ee1"
server_key_version: "encryptionkey@0"
I20260811 22:11:37.558338 12775 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 22:11:37.561795 12808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:37.563233 12775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:11:37.563410 12775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "4d8a01071ab348fda32ace0e88684204"
format_stamp: "Formatted at 2026-08-11 22:11:37 on dist-test-slave-68xk"
server_key: "7bbe9cd3ccdd27c06a347ca64e34cdce"
server_key_iv: "ee4f1e91b947b769a611a8d5ce9a4ee1"
server_key_version: "encryptionkey@0"
I20260811 22:11:37.563614 12775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:37.590535 12775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:37.594911 12775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:37.595206 12775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:37.598292 12775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:37.598397 12775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:37.598580 12775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:37.598707 12775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:11:37 dist-test-slave-68xk krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486297, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:37.622836 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:37.598755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:38949 (local address 127.10.173.190:37173)
0811 22:11:37.599247 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:37.599252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:37.600666 (+ 1414us) server_negotiation.cc:407] Connection header received
0811 22:11:37.602926 (+ 2260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:37.602934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:37.603042 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:37.603181 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:37.605539 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:37.606878 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:37.609350 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:37.609759 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:37.609902 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:37.614903 (+ 5001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:37.614911 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:37.614916 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:37.614965 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:37.618440 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:37.619593 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:37.619600 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:37.619606 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:37.619843 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:37.620336 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:37.620344 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:37.620349 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:37.620644 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:37.620804 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:37.621835 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0811 22:11:37.622197 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":301,"threads_started":1}
I20260811 22:11:37.623668 12823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:37.599547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37173 (local address 127.10.173.129:38949)
0811 22:11:37.600482 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:37.600534 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:37.602548 (+ 2014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:37.603436 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:37.603452 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:37.604304 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:37.605247 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:37.605271 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:37.607053 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:37.607057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:37.609182 (+ 2125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:37.609195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:37.610032 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:37.610040 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:37.610263 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:37.611218 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:37.611261 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:37.614581 (+ 3320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:37.618639 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:37.618646 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:37.618658 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:37.619398 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:37.619974 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:37.619978 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:37.619982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:37.620207 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:37.620784 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:37.620791 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:37.621368 (+ 577us) client_negotiation.cc:769] Sending connection context
0811 22:11:37.621707 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 22:11:37.622290 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":431,"threads_started":1}
I20260811 22:11:37.645433 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:37.629767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:58767 (local address 127.10.173.190:37173)
0811 22:11:37.629918 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:37.629922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:37.630761 (+ 839us) server_negotiation.cc:407] Connection header received
0811 22:11:37.630963 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:37.630971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:37.631082 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:37.631210 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:37.632260 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:37.633692 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:37.635591 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:37.636048 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:37.636221 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:37.638089 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:37.638095 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:37.638098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:37.638131 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:37.641626 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:37.642639 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:37.642647 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:37.642654 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:37.642849 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:37.643411 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:37.643420 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:37.643425 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:37.643890 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:37.644088 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:37.644749 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 22:11:37.644993 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:11:37.645742 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:37.629873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37173 (local address 127.10.173.129:58767)
0811 22:11:37.630627 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:37.630651 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:37.630822 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:37.631273 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:37.631277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:37.631760 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:37.632086 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:37.632096 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:37.633863 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:37.633868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:37.635413 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:37.635426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:37.636258 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:37.636266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:37.636448 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:37.637109 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:37.637128 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:37.637884 (+ 756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:37.641768 (+ 3884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:37.641774 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:37.641779 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:37.642503 (+ 724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:37.643000 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:37.643007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:37.643017 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:37.643228 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:37.644048 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:37.644055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:37.644434 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 22:11:37.644630 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:11:37.644995 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":375,"threads_started":1}
I20260811 22:11:37.656132 12775 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:42759
I20260811 22:11:37.656167 12994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:42759 every 8 connection(s)
I20260811 22:11:37.657389 12775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:11:37.659629 12995 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37173
I20260811 22:11:37.659823 12995 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:37.660441 12995 heartbeater.cc:511] Master 127.10.173.190:37173 requested a full tablet report, sending...
I20260811 22:11:37.662381 12659 ts_manager.cc:194] Registered new tserver with Master: 4d8a01071ab348fda32ace0e88684204 (127.10.173.129:42759)
I20260811 22:11:37.663175 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 12775
I20260811 22:11:37.663399 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:11:37.664016 10934 external_mini_cluster.cc:1428] Setting key 5194b6f9e6f70dea401e568c641ee7e4
I20260811 22:11:37.664220 12659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:38949
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:11:37.710819 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37173
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:38.047950 13000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:38.048223 13000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:38.048381 13000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:38.055470 13000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:38.055625 13000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:38.055779 13000 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:11:38.055919 13000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:11:38.065541 13000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.173.190:37173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:38.066936 13000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:38.068365 13000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:38.081683 13007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:38.081727 13012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:38.081759 13009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:38.082593 13000 server_base.cc:1034] running on GCE node
I20260811 22:11:38.083217 13000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:38.084035 13000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:38.085245 13000 hybrid_clock.cc:648] HybridClock initialized: now 1786486298085186 us; error 39 us; skew 500 ppm
I20260811 22:11:38.085556 13000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:38 dist-test-slave-68xk krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486298, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:38.088961 13000 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:11:38.090587 13000 webserver.cc:460] Webserver started at http://127.10.173.130:42211/ using document root <none> and password file <none>
I20260811 22:11:38.091274 13000 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:38.091385 13000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:38.091775 13000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:38.094718 13000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "371ee7a3162a4f96a4717eca5be30933"
format_stamp: "Formatted at 2026-08-11 22:11:38 on dist-test-slave-68xk"
server_key: "536abfc90c60a431761ce590105827eb"
server_key_iv: "6f6e1062653d38194d3f4a6166bf01ef"
server_key_version: "encryptionkey@0"
I20260811 22:11:38.095525 13000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "371ee7a3162a4f96a4717eca5be30933"
format_stamp: "Formatted at 2026-08-11 22:11:38 on dist-test-slave-68xk"
server_key: "536abfc90c60a431761ce590105827eb"
server_key_iv: "6f6e1062653d38194d3f4a6166bf01ef"
server_key_version: "encryptionkey@0"
I20260811 22:11:38.100665 13000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:11:38.104072 13020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:38.105343 13000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 22:11:38.105504 13000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "371ee7a3162a4f96a4717eca5be30933"
format_stamp: "Formatted at 2026-08-11 22:11:38 on dist-test-slave-68xk"
server_key: "536abfc90c60a431761ce590105827eb"
server_key_iv: "6f6e1062653d38194d3f4a6166bf01ef"
server_key_version: "encryptionkey@0"
I20260811 22:11:38.105716 13000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:38.135300 13000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:38.139853 13000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:38.140151 13000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:38.142957 13000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:38.143076 13000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:38.143239 13000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:38.143316 13000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:11:38 dist-test-slave-68xk krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486298, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:38.166988 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.143810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:53007 (local address 127.10.173.190:37173)
0811 22:11:38.144009 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:38.144015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:38.145528 (+ 1513us) server_negotiation.cc:407] Connection header received
0811 22:11:38.148662 (+ 3134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:38.148668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:38.148766 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:38.148890 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:38.151553 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.152599 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.154229 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.154749 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.154911 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.159433 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:38.159443 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:38.159451 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:38.159509 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:38.163370 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:38.164373 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:38.164380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:38.164383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:38.164558 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:38.165134 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:38.165142 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:38.165147 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:38.165457 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:38.165631 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:38.166323 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 22:11:38.166587 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:11:38.168051 13032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.144426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37173 (local address 127.10.173.130:53007)
0811 22:11:38.145321 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:38.145386 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:38.148335 (+ 2949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:38.149154 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:38.149169 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:38.150165 (+ 996us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:38.151326 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.151359 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.152760 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.152769 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.154070 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.154084 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.155255 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.155261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.155558 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.156427 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:38.156454 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:38.159175 (+ 2721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:38.163510 (+ 4335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:38.163519 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:38.163535 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:38.164205 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:38.164694 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:38.164700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:38.164708 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:38.164990 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:38.165593 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:38.165601 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:38.165955 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 22:11:38.166223 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 22:11:38.166845 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":348,"threads_started":1}
I20260811 22:11:38.190722 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.175660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:39783 (local address 127.10.173.190:37173)
0811 22:11:38.175910 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:38.175918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:38.176523 (+ 605us) server_negotiation.cc:407] Connection header received
0811 22:11:38.176743 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:38.176747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:38.176819 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:38.176901 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:38.178319 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.179255 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.180659 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.181046 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.181177 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.183750 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:38.183759 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:38.183766 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:38.183812 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:38.187282 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:38.188300 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:38.188307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:38.188310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:38.188486 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:38.188976 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:38.188981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:38.188986 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:38.189485 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:38.189647 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:38.190072 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 22:11:38.190290 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:11:38.190877 13125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.175658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37173 (local address 127.10.173.130:39783)
0811 22:11:38.176398 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:38.176424 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:38.176593 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:38.176965 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:38.176972 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:38.177705 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:38.178170 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.178186 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.179405 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.179410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.180525 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.180539 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.181233 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.181240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.181408 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.182141 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:38.182161 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:38.183505 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:38.187417 (+ 3912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:38.187423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:38.187427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:38.188163 (+ 736us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:38.188649 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:38.188658 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:38.188661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:38.188848 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:38.189614 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:38.189618 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:38.189816 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 22:11:38.189977 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 22:11:38.190303 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":382,"threads_started":1}
I20260811 22:11:38.202963 13000 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:39367
I20260811 22:11:38.202996 13207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:39367 every 8 connection(s)
I20260811 22:11:38.204293 13000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:11:38.206656 13210 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37173
I20260811 22:11:38.206914 13210 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:38.207608 13210 heartbeater.cc:511] Master 127.10.173.190:37173 requested a full tablet report, sending...
I20260811 22:11:38.208951 12659 ts_manager.cc:194] Registered new tserver with Master: 371ee7a3162a4f96a4717eca5be30933 (127.10.173.130:39367)
I20260811 22:11:38.209885 12659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:53007
I20260811 22:11:38.211630 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 13000
I20260811 22:11:38.211850 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:11:38.212416 10934 external_mini_cluster.cc:1428] Setting key 794095e3264a8e1b5c36cfba3a720dc1
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:11:38.257962 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37173
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:38.590862 13215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:38.591164 13215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:38.591353 13215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:38.598807 13215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:38.598945 13215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:38.599093 13215 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:11:38.599217 13215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:11:38.609673 13215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.173.190:37173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:38.611047 13215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:38.612355 13215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:38.626186 13222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:38.626212 13225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:38.626132 13228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:38.627388 13215 server_base.cc:1034] running on GCE node
I20260811 22:11:38.627995 13215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:38.628968 13215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:38.630249 13215 hybrid_clock.cc:648] HybridClock initialized: now 1786486298630173 us; error 44 us; skew 500 ppm
I20260811 22:11:38.630620 13215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:38 dist-test-slave-68xk krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486298, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:38.633760 13215 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:11:38.635313 13215 webserver.cc:460] Webserver started at http://127.10.173.131:44141/ using document root <none> and password file <none>
I20260811 22:11:38.636003 13215 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:38.636092 13215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:38.636427 13215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:38.639421 13215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "7662db7619bf46769ad9f33844ed9299"
format_stamp: "Formatted at 2026-08-11 22:11:38 on dist-test-slave-68xk"
server_key: "ac6251a66ce7fa8632453a072ec35ff4"
server_key_iv: "6fd430165aeb44a982434177768ffc07"
server_key_version: "encryptionkey@0"
I20260811 22:11:38.640234 13215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "7662db7619bf46769ad9f33844ed9299"
format_stamp: "Formatted at 2026-08-11 22:11:38 on dist-test-slave-68xk"
server_key: "ac6251a66ce7fa8632453a072ec35ff4"
server_key_iv: "6fd430165aeb44a982434177768ffc07"
server_key_version: "encryptionkey@0"
I20260811 22:11:38.645380 13215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:11:38.648860 13236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:38.650151 13215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:11:38.650331 13215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "7662db7619bf46769ad9f33844ed9299"
format_stamp: "Formatted at 2026-08-11 22:11:38 on dist-test-slave-68xk"
server_key: "ac6251a66ce7fa8632453a072ec35ff4"
server_key_iv: "6fd430165aeb44a982434177768ffc07"
server_key_version: "encryptionkey@0"
I20260811 22:11:38.650610 13215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:38.667130 12995 heartbeater.cc:503] Master 127.10.173.190:37173 was elected leader, sending a full tablet report...
I20260811 22:11:38.669239 13215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:38.674268 13215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:38.674635 13215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:38.677587 13215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:38.677700 13215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:38.677868 13215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:38.677945 13215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:11:38 dist-test-slave-68xk krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486298, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:38.705361 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.678366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:33031 (local address 127.10.173.190:37173)
0811 22:11:38.678604 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:38.678622 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:38.680310 (+ 1688us) server_negotiation.cc:407] Connection header received
0811 22:11:38.683068 (+ 2758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:38.683076 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:38.683187 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:38.683336 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:38.685763 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.687012 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.690287 (+ 3275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.690880 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.691047 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.696328 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:38.696341 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:38.696345 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:38.696380 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:38.700631 (+ 4251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:38.701907 (+ 1276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:38.701913 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:38.701917 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:38.702095 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:38.702925 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:38.702931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:38.702937 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:38.703364 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:38.703573 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:38.704688 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0811 22:11:38.704947 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 22:11:38.706741 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.679073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37173 (local address 127.10.173.131:33031)
0811 22:11:38.680100 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:38.680151 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:38.682603 (+ 2452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:38.683657 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:38.683666 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:38.684527 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:38.685549 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.685585 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.687233 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.687243 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.690086 (+ 2843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.690109 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.691200 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.691211 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.691630 (+ 419us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.692692 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:38.692722 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:38.696010 (+ 3288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:38.700771 (+ 4761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:38.700780 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:38.700797 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:38.701695 (+ 898us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:38.702266 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:38.702271 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:38.702275 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:38.702715 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:38.703565 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:38.703576 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:38.704065 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 22:11:38.704515 (+ 450us) client_negotiation.cc:240] Negotiation successful
0811 22:11:38.705174 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":445,"threads_started":1}
I20260811 22:11:38.728717 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.713033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:50673 (local address 127.10.173.190:37173)
0811 22:11:38.713205 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:38.713209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:38.713857 (+ 648us) server_negotiation.cc:407] Connection header received
0811 22:11:38.713994 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:38.714002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:38.714119 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:38.714246 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:38.715619 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.716480 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.718090 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.718841 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.719002 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.721379 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:38.721389 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:38.721396 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:38.721446 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:38.725325 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:38.726110 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:38.726119 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:38.726126 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:38.726286 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:38.726839 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:38.726847 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:38.726853 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:38.727372 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:38.727586 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:38.728097 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 22:11:38.728334 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:11:38.728828 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.713110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37173 (local address 127.10.173.131:50673)
0811 22:11:38.713740 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:38.713762 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:38.713879 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:38.714264 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:38.714267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:38.714834 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:38.715451 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.715464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.716658 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.716668 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.717948 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.717961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.719073 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.719083 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.719440 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.720146 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:38.720174 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:38.721171 (+ 997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:38.725460 (+ 4289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:38.725464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:38.725467 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:38.725970 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:38.726451 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:38.726460 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:38.726467 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:38.726695 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:38.727535 (+ 840us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:38.727539 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:38.727801 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 22:11:38.727982 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 22:11:38.728340 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":339,"threads_started":1}
I20260811 22:11:38.737689 13215 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:36155
I20260811 22:11:38.737722 13430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:36155 every 8 connection(s)
I20260811 22:11:38.738984 13215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:11:38.741026 13431 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37173
I20260811 22:11:38.741289 13431 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:38.741959 13431 heartbeater.cc:511] Master 127.10.173.190:37173 requested a full tablet report, sending...
I20260811 22:11:38.743389 12659 ts_manager.cc:194] Registered new tserver with Master: 7662db7619bf46769ad9f33844ed9299 (127.10.173.131:36155)
I20260811 22:11:38.744261 12659 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:33031
I20260811 22:11:38.748373 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 13215
I20260811 22:11:38.748571 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:11:38.749004 10934 external_mini_cluster.cc:1428] Setting key 86487b8c46cdd0ac186f102d04e975de
I20260811 22:11:38.751861 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:11:38 dist-test-slave-68xk krb5kdc[12590](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486298, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:11:38 dist-test-slave-68xk krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486298, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:38.803375 12820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.789586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.10.173.190:37173)
0811 22:11:38.789848 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:38.789856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:38.790020 (+ 164us) server_negotiation.cc:407] Connection header received
0811 22:11:38.790242 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:38.790247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:38.790317 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:38.790446 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:38.791880 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.792619 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.793906 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.794349 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.794507 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.798386 (+ 3879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:38.798391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:38.798394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:38.798476 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:38.801078 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:38.801683 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:38.801687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:38.801689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:38.801821 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:38.802139 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:38.802143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:38.802144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:38.802462 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:38.802664 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:38.802923 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 22:11:38.803096 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:11:38.807827 12659 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:11:38.815220 12659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:11:38.839335 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.828443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58784 (local address 127.10.173.129:42759)
0811 22:11:38.831322 (+ 2879us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:38.831332 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:38.831368 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:11:38.831472 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:38.831480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:38.831712 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:38.832615 (+ 903us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:38.832834 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.833869 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.837169 (+ 3300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.838179 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.838370 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.838578 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:38.838860 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 22:11:38.839022 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2710,"thread_start_us":2587,"threads_started":1}
I20260811 22:11:38.839506 13447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.828175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:42759 (local address 127.0.0.1:58784)
0811 22:11:38.828919 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:38.828954 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:38.829117 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:38.831941 (+ 2824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:38.831949 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:38.831970 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:38.832409 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.832423 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.834000 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.834008 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.836996 (+ 2988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.837010 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.838385 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.838392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.838633 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.838679 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:11:38.838804 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:11:38.839034 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":2,"thread_start_us":359,"threads_started":1}
I20260811 22:11:38.840329 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.829068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32914 (local address 127.10.173.131:36155)
0811 22:11:38.829971 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:38.829981 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:38.830274 (+ 293us) server_negotiation.cc:407] Connection header received
0811 22:11:38.830534 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:38.830540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:38.830719 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:38.830971 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:38.832090 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.834301 (+ 2211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.836896 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.838456 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.838649 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.838804 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:38.839846 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0811 22:11:38.840006 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":457,"threads_started":1}
I20260811 22:11:38.840728 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.828319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:36155 (local address 127.0.0.1:32914)
0811 22:11:38.830164 (+ 1845us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:38.830178 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:38.830322 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:38.831347 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:38.831354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:38.831386 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:38.831806 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.831822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.834457 (+ 2635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.834465 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.836725 (+ 2260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.836739 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.838968 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.838976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.839127 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.839160 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:11:38.839252 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:11:38.839396 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1726,"thread_start_us":1601,"threads_started":1}
I20260811 22:11:38.842280 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.831130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:39367 (local address 127.0.0.1:50976)
0811 22:11:38.831738 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:38.831764 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:38.831896 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:38.834997 (+ 3101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:38.835002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:38.835038 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:38.835523 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.835536 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.837296 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.837302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.839387 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:38.839399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.841496 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.841502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:38.841683 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.841716 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:11:38.841807 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:11:38.841943 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":733,"threads_started":1}
I20260811 22:11:38.843012 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:38.831674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50976 (local address 127.10.173.130:39367)
0811 22:11:38.834435 (+ 2761us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:38.834444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:38.834469 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:11:38.834558 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:38.834565 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:38.834786 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:38.834978 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:38.835739 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.837149 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.839578 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:38.841002 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:38.841195 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:38.841359 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:38.842548 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0811 22:11:38.842734 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2573,"thread_start_us":2444,"threads_started":1}
I20260811 22:11:38.844237 13311 tablet_service.cc:1467] Processing CreateTablet for tablet f83ea466b9ff4af08d29df55b6f58485 (DEFAULT_TABLE table=test-table [id=5422c6dd03b44c51a970c0614c38b69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:38.844240 12882 tablet_service.cc:1467] Processing CreateTablet for tablet f83ea466b9ff4af08d29df55b6f58485 (DEFAULT_TABLE table=test-table [id=5422c6dd03b44c51a970c0614c38b69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:38.845597 13311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f83ea466b9ff4af08d29df55b6f58485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:38.845621 12882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f83ea466b9ff4af08d29df55b6f58485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:38.848453 13094 tablet_service.cc:1467] Processing CreateTablet for tablet f83ea466b9ff4af08d29df55b6f58485 (DEFAULT_TABLE table=test-table [id=5422c6dd03b44c51a970c0614c38b69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:38.849879 13094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f83ea466b9ff4af08d29df55b6f58485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:38.857609 13453 tablet_bootstrap.cc:492] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: Bootstrap starting.
I20260811 22:11:38.861102 13454 tablet_bootstrap.cc:492] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: Bootstrap starting.
I20260811 22:11:38.862144 13453 tablet_bootstrap.cc:654] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:38.863816 13453 log.cc:824] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:38.864712 13455 tablet_bootstrap.cc:492] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: Bootstrap starting.
I20260811 22:11:38.865839 13454 tablet_bootstrap.cc:654] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:38.867239 13454 log.cc:824] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:38.868523 13455 tablet_bootstrap.cc:654] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:38.868682 13453 tablet_bootstrap.cc:492] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: No bootstrap required, opened a new log
I20260811 22:11:38.868980 13453 ts_tablet_manager.cc:1397] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260811 22:11:38.869889 13455 log.cc:824] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:38.871418 13454 tablet_bootstrap.cc:492] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: No bootstrap required, opened a new log
I20260811 22:11:38.871723 13454 ts_tablet_manager.cc:1397] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 22:11:38.872596 13455 tablet_bootstrap.cc:492] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: No bootstrap required, opened a new log
I20260811 22:11:38.872859 13455 ts_tablet_manager.cc:1397] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 22:11:38.880488 13455 raft_consensus.cc:348] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } }
I20260811 22:11:38.880501 13453 raft_consensus.cc:348] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } }
I20260811 22:11:38.880893 13455 raft_consensus.cc:374] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:38.880918 13453 raft_consensus.cc:374] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:38.881004 13453 raft_consensus.cc:729] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7662db7619bf46769ad9f33844ed9299, State: Initialized, Role: FOLLOWER
I20260811 22:11:38.880990 13455 raft_consensus.cc:729] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 371ee7a3162a4f96a4717eca5be30933, State: Initialized, Role: FOLLOWER
I20260811 22:11:38.881633 13455 consensus_queue.cc:260] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } }
I20260811 22:11:38.881711 13453 consensus_queue.cc:260] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } }
I20260811 22:11:38.883028 13210 heartbeater.cc:503] Master 127.10.173.190:37173 was elected leader, sending a full tablet report...
I20260811 22:11:38.883540 13455 ts_tablet_manager.cc:1428] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: Time spent starting tablet: real 0.011s user 0.009s sys 0.003s
I20260811 22:11:38.884413 13454 raft_consensus.cc:348] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } }
I20260811 22:11:38.884785 13454 raft_consensus.cc:374] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:38.884903 13454 raft_consensus.cc:729] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8a01071ab348fda32ace0e88684204, State: Initialized, Role: FOLLOWER
I20260811 22:11:38.885524 13454 consensus_queue.cc:260] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } }
I20260811 22:11:38.887552 13431 heartbeater.cc:503] Master 127.10.173.190:37173 was elected leader, sending a full tablet report...
I20260811 22:11:38.888070 13453 ts_tablet_manager.cc:1428] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: Time spent starting tablet: real 0.019s user 0.015s sys 0.005s
I20260811 22:11:38.889722 13454 ts_tablet_manager.cc:1428] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
W20260811 22:11:38.910109 12997 tablet.cc:2369] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:11:38.956419 13211 tablet.cc:2369] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:11:38.990697 13433 tablet.cc:2369] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:11:39.230160 13461 raft_consensus.cc:482] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:39.230671 13461 raft_consensus.cc:504] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } }
I20260811 22:11:39.232729 13461 leader_election.cc:288] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d8a01071ab348fda32ace0e88684204 (127.10.173.129:42759), 7662db7619bf46769ad9f33844ed9299 (127.10.173.131:36155)
I20260811 22:11:39.240342 13466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.233078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:36155 (local address 127.10.173.130:37893)
0811 22:11:39.234031 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:39.234056 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:39.234220 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:39.234622 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:39.234625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:39.234648 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:39.235016 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.235025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.236176 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.236182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.237884 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.237896 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.239429 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.239435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.239673 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.239711 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:11:39.239825 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:11:39.239975 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"thread_start_us":648,"threads_started":1}
I20260811 22:11:39.240350 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.233273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:37893 (local address 127.10.173.131:36155)
0811 22:11:39.233514 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:39.233520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:39.234172 (+ 652us) server_negotiation.cc:407] Connection header received
0811 22:11:39.234356 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:39.234361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:39.234492 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:39.234587 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:39.235198 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.236022 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.238047 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.239201 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.239327 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.239386 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:39.239858 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 22:11:39.240021 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 22:11:39.240881 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.233080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:55813 (local address 127.10.173.129:42759)
0811 22:11:39.233315 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:39.233324 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:39.234000 (+ 676us) server_negotiation.cc:407] Connection header received
0811 22:11:39.234358 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:39.234363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:39.234471 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:39.234541 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:39.235534 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.236378 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.238743 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.239820 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.239977 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.240063 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:39.240479 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 22:11:39.240637 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:11:39.240974 13464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.232968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:42759 (local address 127.10.173.130:55813)
0811 22:11:39.233876 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:39.233893 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:39.234102 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:39.234805 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:39.234808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:39.234829 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:39.235375 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.235389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.236540 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.236548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.238579 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.238603 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.240071 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.240080 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.240281 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.240313 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:11:39.240422 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:11:39.240538 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"thread_start_us":602,"threads_started":1}
I20260811 22:11:39.241642 13342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f83ea466b9ff4af08d29df55b6f58485" candidate_uuid: "371ee7a3162a4f96a4717eca5be30933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7662db7619bf46769ad9f33844ed9299" is_pre_election: true
I20260811 22:11:39.242110 13342 raft_consensus.cc:2454] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 371ee7a3162a4f96a4717eca5be30933 in term 0.
I20260811 22:11:39.242125 12927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f83ea466b9ff4af08d29df55b6f58485" candidate_uuid: "371ee7a3162a4f96a4717eca5be30933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8a01071ab348fda32ace0e88684204" is_pre_election: true
I20260811 22:11:39.242686 12927 raft_consensus.cc:2454] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 371ee7a3162a4f96a4717eca5be30933 in term 0.
I20260811 22:11:39.242739 13024 leader_election.cc:302] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 371ee7a3162a4f96a4717eca5be30933, 7662db7619bf46769ad9f33844ed9299; no voters:
I20260811 22:11:39.243188 13461 raft_consensus.cc:2781] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:39.243357 13461 raft_consensus.cc:482] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:39.243557 13461 raft_consensus.cc:3037] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:39.246042 13461 raft_consensus.cc:504] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } }
I20260811 22:11:39.246745 13461 leader_election.cc:288] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [CANDIDATE]: Term 1 election: Requested vote from peers 4d8a01071ab348fda32ace0e88684204 (127.10.173.129:42759), 7662db7619bf46769ad9f33844ed9299 (127.10.173.131:36155)
I20260811 22:11:39.247123 12927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f83ea466b9ff4af08d29df55b6f58485" candidate_uuid: "371ee7a3162a4f96a4717eca5be30933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8a01071ab348fda32ace0e88684204"
I20260811 22:11:39.247270 13342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f83ea466b9ff4af08d29df55b6f58485" candidate_uuid: "371ee7a3162a4f96a4717eca5be30933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7662db7619bf46769ad9f33844ed9299"
I20260811 22:11:39.247334 12927 raft_consensus.cc:3037] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:39.247505 13342 raft_consensus.cc:3037] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:39.250762 12927 raft_consensus.cc:2454] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 371ee7a3162a4f96a4717eca5be30933 in term 1.
I20260811 22:11:39.250769 13342 raft_consensus.cc:2454] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 371ee7a3162a4f96a4717eca5be30933 in term 1.
I20260811 22:11:39.251171 13024 leader_election.cc:302] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 371ee7a3162a4f96a4717eca5be30933, 7662db7619bf46769ad9f33844ed9299; no voters:
I20260811 22:11:39.251467 13461 raft_consensus.cc:2781] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:39.251917 13461 raft_consensus.cc:686] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 1 LEADER]: Becoming Leader. State: Replica: 371ee7a3162a4f96a4717eca5be30933, State: Running, Role: LEADER
I20260811 22:11:39.252512 13461 consensus_queue.cc:237] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } }
I20260811 22:11:39.257303 12657 catalog_manager.cc:5495] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 reported cstate change: term changed from 0 to 1, leader changed from <none> to 371ee7a3162a4f96a4717eca5be30933 (127.10.173.130). New cstate: current_term: 1 leader_uuid: "371ee7a3162a4f96a4717eca5be30933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } health_report { overall_health: HEALTHY } } }
I20260811 22:11:39.308187 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.302833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50992 (local address 127.10.173.130:39367)
0811 22:11:39.303047 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:39.303052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:39.303073 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:11:39.303427 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:39.303431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:39.303511 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:39.303632 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:39.303990 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.304714 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.306102 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.306573 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.306667 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.307155 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:39.307644 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:39.307759 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:39.307882 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 22:11:39.307948 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:11:39.318318 13342 raft_consensus.cc:1261] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299 [term 1 FOLLOWER]: Refusing update from remote peer 371ee7a3162a4f96a4717eca5be30933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:39.319661 12927 raft_consensus.cc:1261] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204 [term 1 FOLLOWER]: Refusing update from remote peer 371ee7a3162a4f96a4717eca5be30933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:39.319761 13467 consensus_queue.cc:1035] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:39.320538 13461 consensus_queue.cc:1035] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:39.357291 12659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:39.357651 12659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:39.361601 12659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76: Sending DeleteTablet for 3 replicas of tablet f83ea466b9ff4af08d29df55b6f58485
I20260811 22:11:39.363093 13094 tablet_service.cc:1514] Processing DeleteTablet for tablet f83ea466b9ff4af08d29df55b6f58485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:39 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:50976
I20260811 22:11:39.363045 12882 tablet_service.cc:1514] Processing DeleteTablet for tablet f83ea466b9ff4af08d29df55b6f58485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:39 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:58784
I20260811 22:11:39.364288 12659 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:11:39.364802 12659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:11:39.365844 13311 tablet_service.cc:1514] Processing DeleteTablet for tablet f83ea466b9ff4af08d29df55b6f58485 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:39 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:32914
I20260811 22:11:39.366631 13490 tablet_replica.cc:330] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: stopping tablet replica
I20260811 22:11:39.367329 13490 raft_consensus.cc:2229] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:11:39.368257 13490 raft_consensus.cc:2258] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:39.376327 13492 tablet_replica.cc:330] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: stopping tablet replica
I20260811 22:11:39.376602 12881 tablet_service.cc:1467] Processing CreateTablet for tablet 6c01335cadbe41bc8248d75040d92b8b (DEFAULT_TABLE table=test-table [id=f9730e98b9934877b3d80d45dfc71454]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:39.377215 12881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c01335cadbe41bc8248d75040d92b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:39.377799 13490 ts_tablet_manager.cc:1905] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:39.377758 13492 raft_consensus.cc:2229] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:39.378912 13492 raft_consensus.cc:2258] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:39.382727 13311 tablet_service.cc:1467] Processing CreateTablet for tablet 6c01335cadbe41bc8248d75040d92b8b (DEFAULT_TABLE table=test-table [id=f9730e98b9934877b3d80d45dfc71454]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:39.383212 13311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c01335cadbe41bc8248d75040d92b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:39.383204 13094 tablet_service.cc:1467] Processing CreateTablet for tablet 6c01335cadbe41bc8248d75040d92b8b (DEFAULT_TABLE table=test-table [id=f9730e98b9934877b3d80d45dfc71454]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:39.383621 13094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c01335cadbe41bc8248d75040d92b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:39.390565 13454 tablet_bootstrap.cc:492] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204: Bootstrap starting.
I20260811 22:11:39.392540 13454 tablet_bootstrap.cc:654] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:39.396243 13492 ts_tablet_manager.cc:1905] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:39.398569 13490 ts_tablet_manager.cc:1918] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:11:39.398835 13490 log.cc:1196] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/f83ea466b9ff4af08d29df55b6f58485
I20260811 22:11:39.399428 13490 ts_tablet_manager.cc:1939] T f83ea466b9ff4af08d29df55b6f58485 P 371ee7a3162a4f96a4717eca5be30933: Deleting consensus metadata
I20260811 22:11:39.401525 12637 catalog_manager.cc:4841] TS 371ee7a3162a4f96a4717eca5be30933 (127.10.173.130:39367): tablet f83ea466b9ff4af08d29df55b6f58485 (table test-table [id=5422c6dd03b44c51a970c0614c38b69b]) successfully deleted
I20260811 22:11:39.402148 13454 tablet_bootstrap.cc:492] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204: No bootstrap required, opened a new log
I20260811 22:11:39.403080 13489 tablet_replica.cc:330] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: stopping tablet replica
I20260811 22:11:39.403796 13454 ts_tablet_manager.cc:1397] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.004s
I20260811 22:11:39.404062 13489 raft_consensus.cc:2229] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:39.404659 13494 tablet_bootstrap.cc:492] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933: Bootstrap starting.
I20260811 22:11:39.404661 13489 raft_consensus.cc:2258] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:39.405110 13454 raft_consensus.cc:348] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.405409 13454 raft_consensus.cc:374] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:39.405575 13454 raft_consensus.cc:729] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8a01071ab348fda32ace0e88684204, State: Initialized, Role: FOLLOWER
I20260811 22:11:39.405998 13454 consensus_queue.cc:260] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.406837 13494 tablet_bootstrap.cc:654] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:39.406872 13489 ts_tablet_manager.cc:1905] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:39.408730 13454 ts_tablet_manager.cc:1428] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260811 22:11:39.414183 13496 tablet_bootstrap.cc:492] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299: Bootstrap starting.
I20260811 22:11:39.416782 13496 tablet_bootstrap.cc:654] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:39.420048 13492 ts_tablet_manager.cc:1918] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:11:39.423175 13492 log.cc:1196] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/f83ea466b9ff4af08d29df55b6f58485
I20260811 22:11:39.425545 13492 ts_tablet_manager.cc:1939] T f83ea466b9ff4af08d29df55b6f58485 P 7662db7619bf46769ad9f33844ed9299: Deleting consensus metadata
I20260811 22:11:39.428658 12640 catalog_manager.cc:4841] TS 7662db7619bf46769ad9f33844ed9299 (127.10.173.131:36155): tablet f83ea466b9ff4af08d29df55b6f58485 (table test-table [id=5422c6dd03b44c51a970c0614c38b69b]) successfully deleted
I20260811 22:11:39.431097 13489 ts_tablet_manager.cc:1918] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:11:39.432005 13489 log.cc:1196] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/f83ea466b9ff4af08d29df55b6f58485
I20260811 22:11:39.432685 13489 ts_tablet_manager.cc:1939] T f83ea466b9ff4af08d29df55b6f58485 P 4d8a01071ab348fda32ace0e88684204: Deleting consensus metadata
I20260811 22:11:39.432829 13494 tablet_bootstrap.cc:492] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933: No bootstrap required, opened a new log
I20260811 22:11:39.433012 13494 ts_tablet_manager.cc:1397] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933: Time spent bootstrapping tablet: real 0.029s user 0.008s sys 0.012s
I20260811 22:11:39.433969 13496 tablet_bootstrap.cc:492] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299: No bootstrap required, opened a new log
I20260811 22:11:39.433930 13494 raft_consensus.cc:348] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.434136 13496 ts_tablet_manager.cc:1397] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.002s
I20260811 22:11:39.434224 13494 raft_consensus.cc:374] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:39.434871 13494 raft_consensus.cc:729] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 371ee7a3162a4f96a4717eca5be30933, State: Initialized, Role: FOLLOWER
I20260811 22:11:39.434966 13496 raft_consensus.cc:348] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.435242 13496 raft_consensus.cc:374] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:39.435429 13496 raft_consensus.cc:729] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7662db7619bf46769ad9f33844ed9299, State: Initialized, Role: FOLLOWER
I20260811 22:11:39.435405 12641 catalog_manager.cc:4841] TS 4d8a01071ab348fda32ace0e88684204 (127.10.173.129:42759): tablet f83ea466b9ff4af08d29df55b6f58485 (table test-table [id=5422c6dd03b44c51a970c0614c38b69b]) successfully deleted
I20260811 22:11:39.435379 13494 consensus_queue.cc:260] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.437237 13494 ts_tablet_manager.cc:1428] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 22:11:39.438143 13496 consensus_queue.cc:260] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.439471 13496 ts_tablet_manager.cc:1428] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299: Time spent starting tablet: real 0.005s user 0.001s sys 0.002s
I20260811 22:11:39.571722 13462 raft_consensus.cc:482] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:39.572022 13462 raft_consensus.cc:504] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.574064 13462 leader_election.cc:288] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d8a01071ab348fda32ace0e88684204 (127.10.173.129:42759), 371ee7a3162a4f96a4717eca5be30933 (127.10.173.130:39367)
I20260811 22:11:39.583511 13507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.574564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:39367 (local address 127.10.173.131:47561)
0811 22:11:39.576285 (+ 1721us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:39.576304 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:39.576478 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:39.576892 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:39.576897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:39.576926 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:39.577432 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.577445 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.578911 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.578919 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.580527 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.580541 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.582512 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.582520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.582795 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.582842 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 22:11:39.582946 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:11:39.583102 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1584,"mutex_wait_us":42,"thread_start_us":1370,"threads_started":1}
I20260811 22:11:39.583527 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.575250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:47561 (local address 127.10.173.130:39367)
0811 22:11:39.575532 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:39.575541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:39.576412 (+ 871us) server_negotiation.cc:407] Connection header received
0811 22:11:39.576664 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:39.576668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:39.576756 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:39.576868 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:39.577659 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.578759 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.580776 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.582250 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.582457 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.582546 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:39.583006 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 22:11:39.583175 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 22:11:39.583523 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.574535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:54423 (local address 127.10.173.129:42759)
0811 22:11:39.574854 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:39.574865 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:39.575345 (+ 480us) server_negotiation.cc:407] Connection header received
0811 22:11:39.575696 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:39.575703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:39.575784 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:39.575852 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:39.576815 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.577822 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.580710 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.582250 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.582474 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.582547 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:39.583092 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 22:11:39.583236 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260811 22:11:39.584332 13506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.574484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:42759 (local address 127.10.173.131:54423)
0811 22:11:39.575227 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:39.575249 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:39.575472 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:39.576076 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:39.576082 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:39.576114 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:39.576618 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.576633 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.577966 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.577972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.580513 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.580529 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.582721 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.582726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.582870 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.582895 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:11:39.583828 (+ 933us) client_negotiation.cc:240] Negotiation successful
0811 22:11:39.583944 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":66,"thread_start_us":1059,"threads_started":1}
I20260811 22:11:39.585062 12927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c01335cadbe41bc8248d75040d92b8b" candidate_uuid: "7662db7619bf46769ad9f33844ed9299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8a01071ab348fda32ace0e88684204" is_pre_election: true
I20260811 22:11:39.585106 13132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c01335cadbe41bc8248d75040d92b8b" candidate_uuid: "7662db7619bf46769ad9f33844ed9299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "371ee7a3162a4f96a4717eca5be30933" is_pre_election: true
I20260811 22:11:39.585361 12927 raft_consensus.cc:2454] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7662db7619bf46769ad9f33844ed9299 in term 0.
I20260811 22:11:39.585631 13132 raft_consensus.cc:2454] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7662db7619bf46769ad9f33844ed9299 in term 0.
I20260811 22:11:39.585975 13243 leader_election.cc:302] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8a01071ab348fda32ace0e88684204, 7662db7619bf46769ad9f33844ed9299; no voters:
I20260811 22:11:39.586642 13462 raft_consensus.cc:2781] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:39.586786 13462 raft_consensus.cc:482] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:39.586912 13462 raft_consensus.cc:3037] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:39.590521 13462 raft_consensus.cc:504] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.591331 13462 leader_election.cc:288] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [CANDIDATE]: Term 1 election: Requested vote from peers 4d8a01071ab348fda32ace0e88684204 (127.10.173.129:42759), 371ee7a3162a4f96a4717eca5be30933 (127.10.173.130:39367)
I20260811 22:11:39.591956 12927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c01335cadbe41bc8248d75040d92b8b" candidate_uuid: "7662db7619bf46769ad9f33844ed9299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8a01071ab348fda32ace0e88684204"
I20260811 22:11:39.591997 13132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c01335cadbe41bc8248d75040d92b8b" candidate_uuid: "7662db7619bf46769ad9f33844ed9299" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "371ee7a3162a4f96a4717eca5be30933"
I20260811 22:11:39.592197 12927 raft_consensus.cc:3037] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:39.592229 13132 raft_consensus.cc:3037] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:39.595350 13132 raft_consensus.cc:2454] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7662db7619bf46769ad9f33844ed9299 in term 1.
I20260811 22:11:39.595371 12927 raft_consensus.cc:2454] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7662db7619bf46769ad9f33844ed9299 in term 1.
I20260811 22:11:39.595813 13243 leader_election.cc:302] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8a01071ab348fda32ace0e88684204, 7662db7619bf46769ad9f33844ed9299; no voters:
I20260811 22:11:39.596128 13462 raft_consensus.cc:2781] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:39.596710 13462 raft_consensus.cc:686] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 1 LEADER]: Becoming Leader. State: Replica: 7662db7619bf46769ad9f33844ed9299, State: Running, Role: LEADER
I20260811 22:11:39.597399 13462 consensus_queue.cc:237] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.601382 12657 catalog_manager.cc:5495] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7662db7619bf46769ad9f33844ed9299 (127.10.173.131). New cstate: current_term: 1 leader_uuid: "7662db7619bf46769ad9f33844ed9299" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } health_report { overall_health: HEALTHY } } }
Aug 11 22:11:39 dist-test-slave-68xk krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:39.698881 13522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.669409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37936 (local address 127.10.173.190:37173)
0811 22:11:39.682321 (+ 12912us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:39.682327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:39.683097 (+ 770us) server_negotiation.cc:407] Connection header received
0811 22:11:39.683289 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:39.683295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:39.683409 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:39.683562 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:39.685136 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.686336 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.687753 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.688224 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.688366 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.692477 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:39.692483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:39.692486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:39.692516 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:39.695497 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:39.696545 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:39.696549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:39.696551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:39.696671 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:39.697166 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:39.697170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:39.697172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:39.697596 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:39.697743 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:39.698327 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 22:11:39.698573 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12730,"thread_start_us":12936,"threads_started":1}
I20260811 22:11:39.699414 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.669418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37173 (local address 127.0.0.1:37936)
0811 22:11:39.682960 (+ 13542us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:39.682982 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:39.683144 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:39.683606 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:39.683611 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:39.684619 (+ 1008us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:39.684927 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.684943 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.686495 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.686503 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.687610 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.687623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.688400 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.688406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.688613 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.689356 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:39.689373 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:39.692217 (+ 2844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:39.695664 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:39.695684 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:39.695694 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:39.696412 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:39.696797 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:39.696803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:39.696807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:39.697024 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:39.697764 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:39.697777 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:39.698009 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 22:11:39.698240 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 22:11:39.698805 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13405,"mutex_wait_us":182,"thread_start_us":13259,"threads_started":1}
I20260811 22:11:39.706039 12657 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:37936:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:11:39.707924 12657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:11:39.715967 13094 tablet_service.cc:1467] Processing CreateTablet for tablet ebd0737ab1b945b689107071cb6207fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b52aa27e5324471e9c7439228c7fcb81]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:11:39.715945 12881 tablet_service.cc:1467] Processing CreateTablet for tablet ebd0737ab1b945b689107071cb6207fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b52aa27e5324471e9c7439228c7fcb81]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:11:39.716202 13311 tablet_service.cc:1467] Processing CreateTablet for tablet ebd0737ab1b945b689107071cb6207fc (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b52aa27e5324471e9c7439228c7fcb81]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:11:39.716620 12881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebd0737ab1b945b689107071cb6207fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:39.716658 13094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebd0737ab1b945b689107071cb6207fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:39.716718 13311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ebd0737ab1b945b689107071cb6207fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:39.725661 13496 tablet_bootstrap.cc:492] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: Bootstrap starting.
I20260811 22:11:39.726465 13494 tablet_bootstrap.cc:492] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: Bootstrap starting.
I20260811 22:11:39.727950 13494 tablet_bootstrap.cc:654] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:39.727981 13496 tablet_bootstrap.cc:654] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:39.730206 13494 tablet_bootstrap.cc:492] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: No bootstrap required, opened a new log
I20260811 22:11:39.730468 13494 ts_tablet_manager.cc:1397] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 22:11:39.731251 13454 tablet_bootstrap.cc:492] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: Bootstrap starting.
I20260811 22:11:39.731338 13494 raft_consensus.cc:348] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.732270 13494 raft_consensus.cc:374] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:39.732364 13494 raft_consensus.cc:729] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 371ee7a3162a4f96a4717eca5be30933, State: Initialized, Role: FOLLOWER
I20260811 22:11:39.732723 13494 consensus_queue.cc:260] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.733536 13454 tablet_bootstrap.cc:654] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:39.733412 13467 tablet_replica.cc:439] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } }
I20260811 22:11:39.733836 13467 tablet_replica.cc:442] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:39.734928 13494 ts_tablet_manager.cc:1428] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 22:11:39.740286 13454 tablet_bootstrap.cc:492] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: No bootstrap required, opened a new log
I20260811 22:11:39.740604 13454 ts_tablet_manager.cc:1397] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 22:11:39.741518 13454 raft_consensus.cc:348] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.741885 13454 raft_consensus.cc:374] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:39.742031 13454 raft_consensus.cc:729] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8a01071ab348fda32ace0e88684204, State: Initialized, Role: FOLLOWER
I20260811 22:11:39.742662 13454 consensus_queue.cc:260] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.743104 13463 tablet_replica.cc:439] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } }
I20260811 22:11:39.743647 13463 tablet_replica.cc:442] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:39.744143 13454 ts_tablet_manager.cc:1428] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:11:39.747840 13496 tablet_bootstrap.cc:492] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: No bootstrap required, opened a new log
I20260811 22:11:39.748052 13496 ts_tablet_manager.cc:1397] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.003s
I20260811 22:11:39.748788 13496 raft_consensus.cc:348] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.749032 13496 raft_consensus.cc:374] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:39.749177 13496 raft_consensus.cc:729] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7662db7619bf46769ad9f33844ed9299, State: Initialized, Role: FOLLOWER
I20260811 22:11:39.749449 13496 consensus_queue.cc:260] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.750064 13496 ts_tablet_manager.cc:1428] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 22:11:39.749895 13462 tablet_replica.cc:439] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } }
I20260811 22:11:39.750259 13462 tablet_replica.cc:442] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:39.830317 13467 raft_consensus.cc:482] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:39.830717 13467 raft_consensus.cc:504] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.831620 13467 leader_election.cc:288] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d8a01071ab348fda32ace0e88684204 (127.10.173.129:42759), 7662db7619bf46769ad9f33844ed9299 (127.10.173.131:36155)
I20260811 22:11:39.832067 12927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebd0737ab1b945b689107071cb6207fc" candidate_uuid: "371ee7a3162a4f96a4717eca5be30933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8a01071ab348fda32ace0e88684204" is_pre_election: true
I20260811 22:11:39.832358 12927 raft_consensus.cc:2454] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 371ee7a3162a4f96a4717eca5be30933 in term 0.
I20260811 22:11:39.832437 13342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebd0737ab1b945b689107071cb6207fc" candidate_uuid: "371ee7a3162a4f96a4717eca5be30933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7662db7619bf46769ad9f33844ed9299" is_pre_election: true
I20260811 22:11:39.832723 13342 raft_consensus.cc:2454] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 371ee7a3162a4f96a4717eca5be30933 in term 0.
I20260811 22:11:39.832898 13025 leader_election.cc:302] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 371ee7a3162a4f96a4717eca5be30933, 4d8a01071ab348fda32ace0e88684204; no voters:
I20260811 22:11:39.833289 13467 raft_consensus.cc:2781] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:39.833411 13467 raft_consensus.cc:482] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:39.833536 13467 raft_consensus.cc:3037] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:39.837337 13467 raft_consensus.cc:504] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.838563 12927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebd0737ab1b945b689107071cb6207fc" candidate_uuid: "371ee7a3162a4f96a4717eca5be30933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d8a01071ab348fda32ace0e88684204"
I20260811 22:11:39.838656 13467 leader_election.cc:288] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [CANDIDATE]: Term 1 election: Requested vote from peers 4d8a01071ab348fda32ace0e88684204 (127.10.173.129:42759), 7662db7619bf46769ad9f33844ed9299 (127.10.173.131:36155)
I20260811 22:11:39.838815 12927 raft_consensus.cc:3037] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:39.839162 13342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ebd0737ab1b945b689107071cb6207fc" candidate_uuid: "371ee7a3162a4f96a4717eca5be30933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7662db7619bf46769ad9f33844ed9299"
I20260811 22:11:39.839403 13342 raft_consensus.cc:3037] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:39.841864 12927 raft_consensus.cc:2454] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 371ee7a3162a4f96a4717eca5be30933 in term 1.
I20260811 22:11:39.842386 13025 leader_election.cc:302] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 371ee7a3162a4f96a4717eca5be30933, 4d8a01071ab348fda32ace0e88684204; no voters:
I20260811 22:11:39.842736 13467 raft_consensus.cc:2781] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:39.842947 13342 raft_consensus.cc:2454] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 371ee7a3162a4f96a4717eca5be30933 in term 1.
I20260811 22:11:39.843505 13467 raft_consensus.cc:686] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [term 1 LEADER]: Becoming Leader. State: Replica: 371ee7a3162a4f96a4717eca5be30933, State: Running, Role: LEADER
I20260811 22:11:39.843799 13467 consensus_queue.cc:237] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } }
I20260811 22:11:39.845793 13530 tablet_replica.cc:439] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: TxnStatusTablet state changed. Reason: New leader 371ee7a3162a4f96a4717eca5be30933. Latest consensus state: current_term: 1 leader_uuid: "371ee7a3162a4f96a4717eca5be30933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } }
I20260811 22:11:39.846161 13530 tablet_replica.cc:442] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:11:39.847029 13534 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:11:39.846918 12659 catalog_manager.cc:5495] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 reported cstate change: term changed from 0 to 1, leader changed from <none> to 371ee7a3162a4f96a4717eca5be30933 (127.10.173.130). New cstate: current_term: 1 leader_uuid: "371ee7a3162a4f96a4717eca5be30933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } health_report { overall_health: UNKNOWN } } }
I20260811 22:11:39.847375 13534 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:11:39.872565 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.865947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:39367 (local address 127.0.0.1:51008)
0811 22:11:39.866304 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:39.866322 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:39.866533 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:39.867004 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:39.867009 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:39.867022 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:11:39.867271 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.867281 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.868508 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.868515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.870187 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.870204 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.871252 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.871259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.871442 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.871505 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:11:39.872024 (+ 519us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:11:39.872040 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 22:11:39.872107 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:11:39.872233 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 22:11:39.872596 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.865946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51008 (local address 127.10.173.130:39367)
0811 22:11:39.866173 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:39.866180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:39.866403 (+ 223us) server_negotiation.cc:407] Connection header received
0811 22:11:39.866767 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:39.866771 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:39.866849 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:39.866980 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:39.867474 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.868360 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.870330 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.870939 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.871105 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.871709 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:39.871902 (+ 193us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:39.872017 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:39.872174 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 22:11:39.872273 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:11:39.877768 13342 raft_consensus.cc:1261] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299 [term 1 FOLLOWER]: Refusing update from remote peer 371ee7a3162a4f96a4717eca5be30933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:39.877872 12927 raft_consensus.cc:1261] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204 [term 1 FOLLOWER]: Refusing update from remote peer 371ee7a3162a4f96a4717eca5be30933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:39.878526 13530 consensus_queue.cc:1035] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:39.879210 13467 consensus_queue.cc:1035] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:39.886201 13462 tablet_replica.cc:439] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: TxnStatusTablet state changed. Reason: New leader 371ee7a3162a4f96a4717eca5be30933. Latest consensus state: current_term: 1 leader_uuid: "371ee7a3162a4f96a4717eca5be30933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } }
I20260811 22:11:39.886559 13462 tablet_replica.cc:442] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:39.888374 13463 tablet_replica.cc:439] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: TxnStatusTablet state changed. Reason: New leader 371ee7a3162a4f96a4717eca5be30933. Latest consensus state: current_term: 1 leader_uuid: "371ee7a3162a4f96a4717eca5be30933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } }
I20260811 22:11:39.888731 13463 tablet_replica.cc:442] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:39.890724 13538 mvcc.cc:205] Tried to move back new op lower bound from 7317447884292231168 to 7317447884163080192. Current Snapshot: MvccSnapshot[applied={T|T < 7317447884292231168}]
I20260811 22:11:39.894928 13537 mvcc.cc:205] Tried to move back new op lower bound from 7317447884292231168 to 7317447884163080192. Current Snapshot: MvccSnapshot[applied={T|T < 7317447884292231168}]
I20260811 22:11:39.899979 13462 tablet_replica.cc:439] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "371ee7a3162a4f96a4717eca5be30933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } }
I20260811 22:11:39.900333 13462 tablet_replica.cc:442] T ebd0737ab1b945b689107071cb6207fc P 7662db7619bf46769ad9f33844ed9299: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:39.901209 13530 tablet_replica.cc:439] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "371ee7a3162a4f96a4717eca5be30933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } }
I20260811 22:11:39.901568 13530 tablet_replica.cc:442] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:11:39.906656 13542 tablet_replica.cc:439] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "371ee7a3162a4f96a4717eca5be30933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } }
I20260811 22:11:39.907033 13542 tablet_replica.cc:442] T ebd0737ab1b945b689107071cb6207fc P 371ee7a3162a4f96a4717eca5be30933: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:11:39.908361 13540 mvcc.cc:205] Tried to move back new op lower bound from 7317447884292231168 to 7317447884163080192. Current Snapshot: MvccSnapshot[applied={T|T < 7317447884292231168}]
I20260811 22:11:39.910279 13463 tablet_replica.cc:439] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "371ee7a3162a4f96a4717eca5be30933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 } } peers { permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 } } peers { permanent_uuid: "7662db7619bf46769ad9f33844ed9299" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36155 } } }
I20260811 22:11:39.910598 13463 tablet_replica.cc:442] T ebd0737ab1b945b689107071cb6207fc P 4d8a01071ab348fda32ace0e88684204: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:39.921330 13548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.914361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32922 (local address 127.10.173.131:36155)
0811 22:11:39.915019 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:39.915043 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:39.915402 (+ 359us) server_negotiation.cc:407] Connection header received
0811 22:11:39.915646 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:39.915650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:39.915769 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:39.915938 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:39.916492 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.917412 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.919049 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.919552 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.919691 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.920124 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:39.920687 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:39.920849 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:39.920911 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 22:11:39.920994 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":368,"threads_started":1}
I20260811 22:11:39.940337 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.933408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:46503 (local address 127.10.173.130:39367)
0811 22:11:39.933591 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:39.933596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:39.934194 (+ 598us) server_negotiation.cc:407] Connection header received
0811 22:11:39.934352 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:39.934363 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:39.934489 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:39.934592 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:39.935070 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.936328 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.937756 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.938760 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.938904 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.939446 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:39.939624 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:39.939724 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:39.939865 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 22:11:39.939949 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:11:39.940353 13556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:39.933428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:39367 (local address 127.10.173.131:46503)
0811 22:11:39.934081 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:39.934097 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:39.934227 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:39.934604 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:39.934607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:39.934617 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:11:39.934900 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.934908 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.936510 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.936514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.937599 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:39.937612 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:39.938962 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:39.938969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:39.939149 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:39.939227 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:11:39.939727 (+ 500us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:11:39.939736 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:11:39.939808 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 22:11:39.939968 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":309,"threads_started":1}
I20260811 22:11:39.952940 13132 raft_consensus.cc:1261] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933 [term 1 FOLLOWER]: Refusing update from remote peer 7662db7619bf46769ad9f33844ed9299: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:39.953683 12927 raft_consensus.cc:1261] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204 [term 1 FOLLOWER]: Refusing update from remote peer 7662db7619bf46769ad9f33844ed9299: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:39.953737 13511 consensus_queue.cc:1035] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "371ee7a3162a4f96a4717eca5be30933" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:39.954365 13462 consensus_queue.cc:1035] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d8a01071ab348fda32ace0e88684204" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 42759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:40.030130 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:40.021417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:36155 (local address 127.10.173.130:49815)
0811 22:11:40.022665 (+ 1248us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:40.022687 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:40.022916 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:40.023699 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:40.023704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:40.023719 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:11:40.024152 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:40.024171 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.026084 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.026089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:40.027825 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:40.027838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.028792 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.028797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:40.028917 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:40.028978 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:11:40.029551 (+ 573us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:11:40.029564 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 22:11:40.029658 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:11:40.029815 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":911,"thread_start_us":906,"threads_started":1}
I20260811 22:11:40.030109 13548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:40.021583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:49815 (local address 127.10.173.131:36155)
0811 22:11:40.021787 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:40.021795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:40.022774 (+ 979us) server_negotiation.cc:407] Connection header received
0811 22:11:40.023190 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:40.023196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:40.023301 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:40.023370 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:40.024343 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.025506 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.027960 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.028613 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.028752 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:40.029143 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:40.029279 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:40.029374 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:40.029707 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 22:11:40.029792 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:11:40.075801 12657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:40.076071 12657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:40.079988 12657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14c04cbbdcc24bd78b272cf3524edf76: Sending DeleteTablet for 3 replicas of tablet 6c01335cadbe41bc8248d75040d92b8b
I20260811 22:11:40.081085 12881 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c01335cadbe41bc8248d75040d92b8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:40 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:58784
I20260811 22:11:40.081111 13094 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c01335cadbe41bc8248d75040d92b8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:40 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:50976
I20260811 22:11:40.081777 13311 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c01335cadbe41bc8248d75040d92b8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:40 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:32914
I20260811 22:11:40.095522 13579 tablet_replica.cc:330] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933: stopping tablet replica
I20260811 22:11:40.095796 13579 raft_consensus.cc:2229] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:40.096176 13579 raft_consensus.cc:2258] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:40.097508 13579 ts_tablet_manager.cc:1905] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:40.105831 13579 ts_tablet_manager.cc:1918] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:11:40.105991 13579 log.cc:1196] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/6c01335cadbe41bc8248d75040d92b8b
I20260811 22:11:40.106582 13579 ts_tablet_manager.cc:1939] T 6c01335cadbe41bc8248d75040d92b8b P 371ee7a3162a4f96a4717eca5be30933: Deleting consensus metadata
I20260811 22:11:40.107892 12637 catalog_manager.cc:4841] TS 371ee7a3162a4f96a4717eca5be30933 (127.10.173.130:39367): tablet 6c01335cadbe41bc8248d75040d92b8b (table test-table [id=f9730e98b9934877b3d80d45dfc71454]) successfully deleted
I20260811 22:11:40.114960 13581 tablet_replica.cc:330] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299: stopping tablet replica
I20260811 22:11:40.115186 13577 tablet_replica.cc:330] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204: stopping tablet replica
I20260811 22:11:40.115412 13577 raft_consensus.cc:2229] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:40.115542 13581 raft_consensus.cc:2229] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:11:40.115662 13577 raft_consensus.cc:2258] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:40.116945 13577 ts_tablet_manager.cc:1905] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:40.118522 13581 raft_consensus.cc:2258] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:40.122707 13581 ts_tablet_manager.cc:1905] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:40.130209 13577 ts_tablet_manager.cc:1918] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:11:40.135218 13581 ts_tablet_manager.cc:1918] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:11:40.135457 13577 log.cc:1196] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/6c01335cadbe41bc8248d75040d92b8b
I20260811 22:11:40.136056 13577 ts_tablet_manager.cc:1939] T 6c01335cadbe41bc8248d75040d92b8b P 4d8a01071ab348fda32ace0e88684204: Deleting consensus metadata
I20260811 22:11:40.137359 12641 catalog_manager.cc:4841] TS 4d8a01071ab348fda32ace0e88684204 (127.10.173.129:42759): tablet 6c01335cadbe41bc8248d75040d92b8b (table test-table [id=f9730e98b9934877b3d80d45dfc71454]) successfully deleted
I20260811 22:11:40.137826 13581 log.cc:1196] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/6c01335cadbe41bc8248d75040d92b8b
I20260811 22:11:40.138435 13581 ts_tablet_manager.cc:1939] T 6c01335cadbe41bc8248d75040d92b8b P 7662db7619bf46769ad9f33844ed9299: Deleting consensus metadata
I20260811 22:11:40.140060 12640 catalog_manager.cc:4841] TS 7662db7619bf46769ad9f33844ed9299 (127.10.173.131:36155): tablet 6c01335cadbe41bc8248d75040d92b8b (table test-table [id=f9730e98b9934877b3d80d45dfc71454]) successfully deleted
Aug 11 22:11:40 dist-test-slave-68xk krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486298, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:11:40.156505 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:40.117035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58790 (local address 127.10.173.129:42759)
0811 22:11:40.124300 (+ 7265us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:40.124309 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:40.135324 (+ 11015us) server_negotiation.cc:407] Connection header received
0811 22:11:40.135750 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:40.135757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:40.135942 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:40.136035 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:40.138437 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.139777 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.141758 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.142518 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.142640 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:40.148440 (+ 5800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:40.148458 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:40.148481 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:40.148527 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:40.152980 (+ 4453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:40.153992 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:40.154003 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:40.154007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:40.154170 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:40.154687 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:40.154692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:40.154695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:40.155241 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:40.155613 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:40.155856 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 22:11:40.156097 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7057,"thread_start_us":6904,"threads_started":1}
W20260811 22:11:40.157394 12990 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58790
I20260811 22:11:40.183434 13522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:40.161483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37942 (local address 127.10.173.190:37173)
0811 22:11:40.161714 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:40.161720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:40.166849 (+ 5129us) server_negotiation.cc:407] Connection header received
0811 22:11:40.167012 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:40.167019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:40.167125 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:40.167235 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:40.169087 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.170272 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.172236 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.172944 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.173080 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:40.176315 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:40.176323 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:40.176329 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:40.176374 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:40.180139 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:40.180984 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:40.180989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:40.180994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:40.181160 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:40.181664 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:40.181670 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:40.181673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:40.182052 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:40.182323 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:40.182756 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 22:11:40.183008 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260811 22:11:40.184130 12657 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37942
I20260811 22:11:40.186529 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12775
I20260811 22:11:40.206687 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:40.190033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58792 (local address 127.10.173.129:42759)
0811 22:11:40.190256 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:40.190262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:40.190282 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:11:40.190476 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:40.190482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:40.190570 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:40.190647 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:40.192423 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.193537 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.195403 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.196066 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.196207 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:40.199152 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:40.199159 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:40.199166 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:40.199206 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:40.202813 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:40.203590 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:40.203595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:40.203598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:40.203747 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:40.204334 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:40.204339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:40.204342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:40.204670 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:40.204901 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:40.206066 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0811 22:11:40.206343 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 22:11:40.207703 12990 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58792
W20260811 22:11:40.208395 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12775: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:11:40.208586 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 12775
I20260811 22:11:40.247218 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13000
Aug 11 22:11:40 dist-test-slave-68xk krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486298, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:11:40.269999 13452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:40.248883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.10.173.130:39367)
0811 22:11:40.249075 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:40.249082 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:40.249102 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:11:40.249165 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:40.249172 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:40.249245 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:40.249366 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:40.250949 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.252174 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.254077 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.254880 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.254983 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:40.260801 (+ 5818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:40.260822 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:40.260843 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:40.260886 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:40.265628 (+ 4742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:40.266602 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:40.266612 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:40.266615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:40.266782 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:40.267268 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:40.267274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:40.267284 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:40.267791 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:40.268100 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:40.269286 (+ 1186us) server_negotiation.cc:299] Negotiation successful
0811 22:11:40.269581 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 22:11:40.271008 13200 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51024
W20260811 22:11:40.271744 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13000: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:11:40.271929 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 13000
I20260811 22:11:40.310187 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13215
Aug 11 22:11:40 dist-test-slave-68xk krb5kdc[12590](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486298, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:11:40.328274 13548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:40.311367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.10.173.131:36155)
0811 22:11:40.311583 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:40.311587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:40.311607 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:11:40.311712 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:40.311716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:40.311780 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:40.311915 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:40.313177 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.313988 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.315388 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:40.315906 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:40.316053 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:40.320795 (+ 4742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:40.320815 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:40.320839 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:40.320887 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:40.325089 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:40.325828 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:40.325838 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:40.325842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:40.326035 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:40.326510 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:40.326516 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:40.326519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:40.327007 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:40.327343 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:40.327528 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 22:11:40.327773 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 22:11:40.329075 13421 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32936
W20260811 22:11:40.329789 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13215: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:11:40.329942 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 13215
I20260811 22:11:40.358760 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12613
I20260811 22:11:40.360184 12717 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:40.476770 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 12613
2026-08-11T22:11:40Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5556 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:11:40 dist-test-slave-68xk krb5kdc[13600](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:11:40 dist-test-slave-68xk krb5kdc[13600](info): set up 2 sockets
Aug 11 22:11:40 dist-test-slave-68xk krb5kdc[13600](info): commencing operation
krb5kdc: starting...
W20260811 22:11:41.794795 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:11:41 dist-test-slave-68xk krb5kdc[13600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:11:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:11:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:11:42.015046 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41811
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:37547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:41811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:42.352634 13616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:42.352931 13616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:42.353021 13616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:42.360795 13616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:11:42.360910 13616 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:11:42.360951 13616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:42.361013 13616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:42.361069 13616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:11:42.361131 13616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:11:42.372414 13616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:41811
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:42.373700 13616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:42.375120 13616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:42.387178 13627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:42.387213 13629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:42.387909 13632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:42.388229 13616 server_base.cc:1034] running on GCE node
I20260811 22:11:42.388978 13616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:42.390384 13616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:42.391682 13616 hybrid_clock.cc:648] HybridClock initialized: now 1786486302391620 us; error 41 us; skew 500 ppm
I20260811 22:11:42.392016 13616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:42 dist-test-slave-68xk krb5kdc[13600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486302, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:42.398805 13616 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:11:42.400322 13616 webserver.cc:460] Webserver started at http://127.10.173.190:37981/ using document root <none> and password file <none>
I20260811 22:11:42.401034 13616 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:42.401185 13616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:42.401634 13616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:42.404875 13616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "c0f1844ac4f14edeaa99b9e782aedf44"
format_stamp: "Formatted at 2026-08-11 22:11:42 on dist-test-slave-68xk"
server_key: "d8145f3fa9086095adc39ea9f7f9814d"
server_key_iv: "082f0a75ce8b80e82f72e09afee2e287"
server_key_version: "encryptionkey@0"
I20260811 22:11:42.405709 13616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "c0f1844ac4f14edeaa99b9e782aedf44"
format_stamp: "Formatted at 2026-08-11 22:11:42 on dist-test-slave-68xk"
server_key: "d8145f3fa9086095adc39ea9f7f9814d"
server_key_iv: "082f0a75ce8b80e82f72e09afee2e287"
server_key_version: "encryptionkey@0"
I20260811 22:11:42.411029 13616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 22:11:42.414645 13642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:42.416038 13616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:11:42.416184 13616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "c0f1844ac4f14edeaa99b9e782aedf44"
format_stamp: "Formatted at 2026-08-11 22:11:42 on dist-test-slave-68xk"
server_key: "d8145f3fa9086095adc39ea9f7f9814d"
server_key_iv: "082f0a75ce8b80e82f72e09afee2e287"
server_key_version: "encryptionkey@0"
I20260811 22:11:42.416361 13616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:42.450268 13616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:42.454627 13616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:42.454851 13616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:42.476634 13616 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:41811
I20260811 22:11:42.476655 13735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:41811 every 8 connection(s)
I20260811 22:11:42.477929 13616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:11:42.481702 13736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:42.486513 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 13616
I20260811 22:11:42.486680 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:11:42.487285 10934 external_mini_cluster.cc:1428] Setting key f23e751583224abf87e9b483ddd3ab67
I20260811 22:11:42.493934 13736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44: Bootstrap starting.
I20260811 22:11:42.497756 13736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:42.498926 13736 log.cc:824] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:42.501837 13736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44: No bootstrap required, opened a new log
Aug 11 22:11:42 dist-test-slave-68xk krb5kdc[13600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:42.511106 13736 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f1844ac4f14edeaa99b9e782aedf44" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41811 } }
I20260811 22:11:42.511584 13736 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:42.511719 13736 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f1844ac4f14edeaa99b9e782aedf44, State: Initialized, Role: FOLLOWER
I20260811 22:11:42.512400 13739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:42.489330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.10.173.190:41811)
0811 22:11:42.490203 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:42.490218 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:42.490276 (+ 58us) server_negotiation.cc:407] Connection header received
0811 22:11:42.492794 (+ 2518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:42.492832 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:42.493304 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:42.493767 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:42.495244 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:42.497356 (+ 2112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:42.498750 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:42.499815 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:42.499955 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:42.504724 (+ 4769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:42.504732 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:42.504747 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:42.504792 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:42.508121 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:42.508976 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:42.508983 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:42.508989 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:42.509181 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:42.509582 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:42.509585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:42.509587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:42.510129 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:42.510357 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:42.510859 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 22:11:42.511176 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":339,"threads_started":1}
I20260811 22:11:42.512331 13736 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f1844ac4f14edeaa99b9e782aedf44" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41811 } }
I20260811 22:11:42.512605 13736 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:11:42.512748 13736 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:11:42.512917 13736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:42.515452 13736 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f1844ac4f14edeaa99b9e782aedf44" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41811 } }
I20260811 22:11:42.515947 13736 leader_election.cc:302] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0f1844ac4f14edeaa99b9e782aedf44; no voters:
I20260811 22:11:42.516532 13736 leader_election.cc:288] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:11:42.516677 13746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:42.517511 13746 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [term 1 LEADER]: Becoming Leader. State: Replica: c0f1844ac4f14edeaa99b9e782aedf44, State: Running, Role: LEADER
I20260811 22:11:42.518002 13746 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f1844ac4f14edeaa99b9e782aedf44" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41811 } }
I20260811 22:11:42.518579 13736 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:11:42.521744 13749 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0f1844ac4f14edeaa99b9e782aedf44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f1844ac4f14edeaa99b9e782aedf44" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41811 } } }
I20260811 22:11:42.521613 13750 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0f1844ac4f14edeaa99b9e782aedf44. Latest consensus state: current_term: 1 leader_uuid: "c0f1844ac4f14edeaa99b9e782aedf44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f1844ac4f14edeaa99b9e782aedf44" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41811 } } }
I20260811 22:11:42.522141 13749 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:42.522149 13750 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44 [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:42.524633 13758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:11:42.529807 13758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:11:42.539670 13758 catalog_manager.cc:1347] Generated new cluster ID: 33e0f45a7f9449388cb7dcb77d4cf683
I20260811 22:11:42.539798 13758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:11:42.566955 13758 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:11:42.568384 13758 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:11:42.581996 13758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44: Generated new TSK 0
I20260811 22:11:42.583166 13758 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:11:42.646277 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41811
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:37547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:42.988569 13786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:42.988840 13786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:42.989001 13786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:42.996282 13786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:42.996398 13786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:42.996569 13786 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:11:42.996634 13786 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:11:42.996739 13786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:11:43.008659 13786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.173.190:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:43.010529 13786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:43.012466 13786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:43.026376 13798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:43.026538 13791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:43.026564 13796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:43.028066 13786 server_base.cc:1034] running on GCE node
I20260811 22:11:43.028733 13786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:43.029718 13786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:43.030962 13786 hybrid_clock.cc:648] HybridClock initialized: now 1786486303030911 us; error 58 us; skew 500 ppm
I20260811 22:11:43.031330 13786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:43 dist-test-slave-68xk krb5kdc[13600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:43.034706 13786 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:11:43.036245 13786 webserver.cc:460] Webserver started at http://127.10.173.129:45359/ using document root <none> and password file <none>
I20260811 22:11:43.036949 13786 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:43.037091 13786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:43.037415 13786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:43.040445 13786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "722b01b4abad48a9ab3f2920c74be774"
format_stamp: "Formatted at 2026-08-11 22:11:43 on dist-test-slave-68xk"
server_key: "13c70adfbaf3ff8ba3f2ad24495bec38"
server_key_iv: "eaf057a919da7003114cbe14560bd847"
server_key_version: "encryptionkey@0"
I20260811 22:11:43.041218 13786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "722b01b4abad48a9ab3f2920c74be774"
format_stamp: "Formatted at 2026-08-11 22:11:43 on dist-test-slave-68xk"
server_key: "13c70adfbaf3ff8ba3f2ad24495bec38"
server_key_iv: "eaf057a919da7003114cbe14560bd847"
server_key_version: "encryptionkey@0"
I20260811 22:11:43.047042 13786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 22:11:43.050485 13809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:43.051805 13786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:11:43.051993 13786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "722b01b4abad48a9ab3f2920c74be774"
format_stamp: "Formatted at 2026-08-11 22:11:43 on dist-test-slave-68xk"
server_key: "13c70adfbaf3ff8ba3f2ad24495bec38"
server_key_iv: "eaf057a919da7003114cbe14560bd847"
server_key_version: "encryptionkey@0"
I20260811 22:11:43.052235 13786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:43.077087 13786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:43.081918 13786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:43.082223 13786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:43.085013 13786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:43.085176 13786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:43.085304 13786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:43.085395 13786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:11:43 dist-test-slave-68xk krb5kdc[13600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:43.108765 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:43.085838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:47447 (local address 127.10.173.190:41811)
0811 22:11:43.086370 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:43.086375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:43.087836 (+ 1461us) server_negotiation.cc:407] Connection header received
0811 22:11:43.089904 (+ 2068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:43.089911 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:43.090013 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:43.090129 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:43.092348 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.093069 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.094734 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.095340 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.095497 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:43.101074 (+ 5577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:43.101083 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:43.101088 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:43.101132 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:43.105134 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:43.106143 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:43.106147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:43.106150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:43.106265 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:43.106771 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:43.106780 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:43.106783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:43.107042 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:43.107222 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:43.108323 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0811 22:11:43.108501 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":317,"threads_started":1}
I20260811 22:11:43.110968 13828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:43.086793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41811 (local address 127.10.173.129:47447)
0811 22:11:43.087669 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:43.087713 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:43.089613 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:43.090437 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:43.090454 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:43.091260 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:43.092164 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:43.092191 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.093233 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.093243 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:43.094587 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:43.094599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.095637 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.095652 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:43.096026 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:43.096895 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:43.096949 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:43.100772 (+ 3823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:43.105317 (+ 4545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:43.105332 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:43.105354 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:43.105960 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:43.106383 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:43.106392 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:43.106395 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:43.106649 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:43.107213 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:43.107226 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:43.107820 (+ 594us) client_negotiation.cc:769] Sending connection context
0811 22:11:43.108197 (+ 377us) client_negotiation.cc:240] Negotiation successful
0811 22:11:43.108911 (+ 714us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":373,"threads_started":1}
I20260811 22:11:43.138087 13827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:43.121631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:51507 (local address 127.10.173.190:41811)
0811 22:11:43.121903 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:43.121912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:43.122556 (+ 644us) server_negotiation.cc:407] Connection header received
0811 22:11:43.122799 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:43.122805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:43.122900 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:43.123008 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:43.124528 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.125246 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.127920 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.128602 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.128741 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:43.131676 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:43.131681 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:43.131684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:43.131720 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:43.134993 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:43.135882 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:43.135885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:43.135888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:43.136014 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:43.136490 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:43.136493 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:43.136495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:43.136742 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:43.136858 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:43.137435 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 22:11:43.137628 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:11:43.138464 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:43.121719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41811 (local address 127.10.173.129:51507)
0811 22:11:43.122406 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:43.122460 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:43.122641 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:43.123058 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:43.123064 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:43.123858 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:43.124352 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:43.124369 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.125399 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.125408 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:43.127676 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:43.127695 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.128883 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.128892 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:43.129187 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:43.129984 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:43.130028 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:43.131479 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:43.135152 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:43.135159 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:43.135164 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:43.135749 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:43.136140 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:43.136148 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:43.136155 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:43.136366 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:43.136858 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:43.136865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:43.137132 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 22:11:43.137340 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 22:11:43.137797 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":368,"threads_started":1}
I20260811 22:11:43.145884 13786 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:38841
I20260811 22:11:43.145928 13999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:38841 every 8 connection(s)
I20260811 22:11:43.147145 13786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:11:43.148346 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 13786
I20260811 22:11:43.148556 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:11:43.149272 10934 external_mini_cluster.cc:1428] Setting key 39ed20f590d9d5a189d8870e6371c612
I20260811 22:11:43.149310 14001 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41811
I20260811 22:11:43.149559 14001 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:43.150250 14001 heartbeater.cc:511] Master 127.10.173.190:41811 requested a full tablet report, sending...
I20260811 22:11:43.152204 13672 ts_manager.cc:194] Registered new tserver with Master: 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841)
I20260811 22:11:43.153627 13672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:47447
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:11:43.201541 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41811
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:37547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:43.550084 14007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:43.550377 14007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:43.550551 14007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:43.558018 14007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:43.558140 14007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:43.558226 14007 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:11:43.558283 14007 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:11:43.558346 14007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:11:43.568252 14007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.173.190:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:43.569483 14007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:43.570858 14007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:43.584412 14016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:43.584630 14015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:43.584421 14019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:43.585333 14007 server_base.cc:1034] running on GCE node
I20260811 22:11:43.585928 14007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:43.586907 14007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:43.588138 14007 hybrid_clock.cc:648] HybridClock initialized: now 1786486303588068 us; error 66 us; skew 500 ppm
I20260811 22:11:43.588465 14007 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:43 dist-test-slave-68xk krb5kdc[13600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:43.591892 14007 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:11:43.593473 14007 webserver.cc:460] Webserver started at http://127.10.173.130:32783/ using document root <none> and password file <none>
I20260811 22:11:43.594180 14007 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:43.594331 14007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:43.594698 14007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:43.598341 14007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "17cea309ae45494e8b9f0ccd8319b47b"
format_stamp: "Formatted at 2026-08-11 22:11:43 on dist-test-slave-68xk"
server_key: "2ccdeb3ad13e9a0a8b0de0e1f2a04009"
server_key_iv: "4efb023d41c3d85a2596d954c6d58326"
server_key_version: "encryptionkey@0"
I20260811 22:11:43.599298 14007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "17cea309ae45494e8b9f0ccd8319b47b"
format_stamp: "Formatted at 2026-08-11 22:11:43 on dist-test-slave-68xk"
server_key: "2ccdeb3ad13e9a0a8b0de0e1f2a04009"
server_key_iv: "4efb023d41c3d85a2596d954c6d58326"
server_key_version: "encryptionkey@0"
I20260811 22:11:43.605115 14007 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 22:11:43.608779 14029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:43.610306 14007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:11:43.610569 14007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "17cea309ae45494e8b9f0ccd8319b47b"
format_stamp: "Formatted at 2026-08-11 22:11:43 on dist-test-slave-68xk"
server_key: "2ccdeb3ad13e9a0a8b0de0e1f2a04009"
server_key_iv: "4efb023d41c3d85a2596d954c6d58326"
server_key_version: "encryptionkey@0"
I20260811 22:11:43.610796 14007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:43.631107 14007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:43.636076 14007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:43.636420 14007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:43.639590 14007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:43.639755 14007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:43.639874 14007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:43.639935 14007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:11:43 dist-test-slave-68xk krb5kdc[13600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486303, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:43.663998 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:43.639923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:43619 (local address 127.10.173.190:41811)
0811 22:11:43.640819 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:43.640826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:43.641812 (+ 986us) server_negotiation.cc:407] Connection header received
0811 22:11:43.644145 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:43.644154 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:43.644249 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:43.644382 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:43.646518 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.647896 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.651395 (+ 3499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.651893 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.652054 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:43.657376 (+ 5322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:43.657381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:43.657384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:43.657441 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:43.660466 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:43.661176 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:43.661181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:43.661185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:43.661312 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:43.661773 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:43.661778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:43.661781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:43.662053 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:43.662210 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:43.663094 (+ 884us) server_negotiation.cc:299] Negotiation successful
0811 22:11:43.663411 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":641,"threads_started":1}
I20260811 22:11:43.664608 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:43.640677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41811 (local address 127.10.173.130:43619)
0811 22:11:43.641627 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:43.641682 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:43.643861 (+ 2179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:43.644621 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:43.644629 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:43.645391 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:43.646246 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:43.646294 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.648138 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.648147 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:43.651195 (+ 3048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:43.651219 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.652418 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.652425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:43.652686 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:43.653539 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:43.653579 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:43.657104 (+ 3525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:43.660605 (+ 3501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:43.660614 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:43.660630 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:43.661046 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:43.661425 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:43.661430 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:43.661433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:43.661646 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:43.662193 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:43.662201 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:43.662614 (+ 413us) client_negotiation.cc:769] Sending connection context
0811 22:11:43.662932 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 22:11:43.663419 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":406,"threads_started":1}
I20260811 22:11:43.687582 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:43.670695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:34965 (local address 127.10.173.190:41811)
0811 22:11:43.670896 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:43.670904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:43.671768 (+ 864us) server_negotiation.cc:407] Connection header received
0811 22:11:43.672098 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:43.672107 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:43.672199 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:43.672312 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:43.673702 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.674484 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.677196 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.677933 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.678079 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:43.680964 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:43.680973 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:43.680979 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:43.681031 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:43.683967 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:43.684793 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:43.684800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:43.684806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:43.685073 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:43.685714 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:43.685722 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:43.685726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:43.686262 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:43.686474 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:43.687179 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 22:11:43.687356 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:11:43.688314 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:43.670762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41811 (local address 127.10.173.130:34965)
0811 22:11:43.671624 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:43.671644 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:43.671825 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:43.672382 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:43.672388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:43.673058 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:43.673547 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:43.673559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.674734 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.674742 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:43.677007 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:43.677031 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:43.678133 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:43.678140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:43.678311 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:43.679162 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:43.679190 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:43.680691 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:43.684098 (+ 3407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:43.684104 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:43.684110 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:43.684677 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:43.685238 (+ 561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:43.685245 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:43.685251 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:43.685529 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:43.686539 (+ 1010us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:43.686548 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:43.686878 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 22:11:43.687084 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 22:11:43.687598 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":528,"threads_started":1}
I20260811 22:11:43.693101 14007 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:35447
I20260811 22:11:43.693152 14218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:35447 every 8 connection(s)
I20260811 22:11:43.694343 14007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:11:43.696375 14221 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41811
I20260811 22:11:43.696601 14221 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:43.697242 14221 heartbeater.cc:511] Master 127.10.173.190:41811 requested a full tablet report, sending...
I20260811 22:11:43.698562 13672 ts_manager.cc:194] Registered new tserver with Master: 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447)
I20260811 22:11:43.699337 13672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:43619
I20260811 22:11:43.702809 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 14007
I20260811 22:11:43.702996 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:11:43.703436 10934 external_mini_cluster.cc:1428] Setting key 06e7c110fb14b020a127cacbd88a6a23
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:11:43.751044 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41811
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:37547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:44.108654 14234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:44.108928 14234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:44.109036 14234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:44.116434 14234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:44.116554 14234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:44.116639 14234 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:11:44.116685 14234 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:11:44.116747 14234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:11:44.126897 14234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.173.190:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:44.128235 14234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:44.129519 14234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:44.142763 14243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:44.142834 14239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:44.142866 14240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:44.144037 14234 server_base.cc:1034] running on GCE node
I20260811 22:11:44.144688 14234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:44.149428 14234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:44.150650 14234 hybrid_clock.cc:648] HybridClock initialized: now 1786486304150588 us; error 62 us; skew 500 ppm
I20260811 22:11:44.150965 14234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:44 dist-test-slave-68xk krb5kdc[13600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486304, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:44.154263 14234 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:11:44.155870 14234 webserver.cc:460] Webserver started at http://127.10.173.131:40723/ using document root <none> and password file <none>
I20260811 22:11:44.156507 14234 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:44.156642 14234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:44.156985 14234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:44.156962 14001 heartbeater.cc:503] Master 127.10.173.190:41811 was elected leader, sending a full tablet report...
I20260811 22:11:44.159904 14234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "27a482b2bfbc401eaf888d9fc560eab3"
format_stamp: "Formatted at 2026-08-11 22:11:44 on dist-test-slave-68xk"
server_key: "666558336990f843d45c5ec399a85af7"
server_key_iv: "89e649884fedbda4bfe656040f895e2d"
server_key_version: "encryptionkey@0"
I20260811 22:11:44.160709 14234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "27a482b2bfbc401eaf888d9fc560eab3"
format_stamp: "Formatted at 2026-08-11 22:11:44 on dist-test-slave-68xk"
server_key: "666558336990f843d45c5ec399a85af7"
server_key_iv: "89e649884fedbda4bfe656040f895e2d"
server_key_version: "encryptionkey@0"
I20260811 22:11:44.165998 14234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 22:11:44.169718 14254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:44.171134 14234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:11:44.171264 14234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "27a482b2bfbc401eaf888d9fc560eab3"
format_stamp: "Formatted at 2026-08-11 22:11:44 on dist-test-slave-68xk"
server_key: "666558336990f843d45c5ec399a85af7"
server_key_iv: "89e649884fedbda4bfe656040f895e2d"
server_key_version: "encryptionkey@0"
I20260811 22:11:44.171429 14234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:44.188573 14234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:44.193099 14234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:44.193418 14234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:44.196475 14234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:44.196600 14234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:44.196731 14234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:44.196835 14234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:11:44 dist-test-slave-68xk krb5kdc[13600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486304, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:44.222321 14266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.197031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:51579 (local address 127.10.173.190:41811)
0811 22:11:44.197890 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.197895 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.198843 (+ 948us) server_negotiation.cc:407] Connection header received
0811 22:11:44.201183 (+ 2340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.201189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.201287 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.201412 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:44.203892 (+ 2480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.205146 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.208067 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.208653 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.208808 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.215099 (+ 6291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:44.215105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:44.215109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:44.215143 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:44.218226 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:44.219275 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:44.219285 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:44.219289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:44.219498 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:44.220207 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:44.220215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:44.220219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:44.220536 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:44.220731 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.221637 (+ 906us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.221876 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":628,"threads_started":1}
I20260811 22:11:44.223934 14267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.197717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41811 (local address 127.10.173.131:51579)
0811 22:11:44.198676 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:44.198717 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:44.200882 (+ 2165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:44.201665 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:44.201673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:44.202533 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:44.203593 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.203626 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.205313 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.205321 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.207906 (+ 2585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.207920 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.209171 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.209182 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.209515 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.210472 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:44.210517 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:44.214824 (+ 4307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:44.218379 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:44.218389 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:44.218405 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:44.219092 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:44.219668 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:44.219671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:44.219674 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:44.219961 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:44.220704 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:44.220713 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:44.221212 (+ 499us) client_negotiation.cc:769] Sending connection context
0811 22:11:44.221507 (+ 295us) client_negotiation.cc:240] Negotiation successful
0811 22:11:44.222094 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":412,"threads_started":1}
I20260811 22:11:44.248070 14266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.231618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:34875 (local address 127.10.173.190:41811)
0811 22:11:44.231808 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.231816 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.232718 (+ 902us) server_negotiation.cc:407] Connection header received
0811 22:11:44.232947 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.232952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.233023 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.233134 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:44.234879 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.236206 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.237673 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.238570 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.238724 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.240788 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:44.240798 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:44.240806 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:44.240859 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:44.244871 (+ 4012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:44.245766 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:44.245775 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:44.245781 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:44.245985 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:44.246483 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:44.246492 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:44.246498 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:44.246796 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:44.246965 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.247415 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.247667 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:11:44.248215 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.231595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41811 (local address 127.10.173.131:34875)
0811 22:11:44.232605 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:44.232626 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:44.232809 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:44.233168 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:44.233173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:44.234022 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:44.234606 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.234628 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.236385 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.236390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.237500 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.237513 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.238829 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.238836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.238984 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.239722 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:44.239740 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:44.240549 (+ 809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:44.245022 (+ 4473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:44.245027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:44.245041 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:44.245638 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:44.246170 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:44.246177 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:44.246182 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:44.246335 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:44.246922 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:44.246930 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:44.247149 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 22:11:44.247304 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 22:11:44.247691 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":628,"threads_started":1}
I20260811 22:11:44.262212 14234 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:41581
I20260811 22:11:44.262236 14440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:41581 every 8 connection(s)
I20260811 22:11:44.263540 14234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:11:44.265841 14441 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41811
I20260811 22:11:44.266170 14441 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:44.266989 14441 heartbeater.cc:511] Master 127.10.173.190:41811 requested a full tablet report, sending...
I20260811 22:11:44.268532 13672 ts_manager.cc:194] Registered new tserver with Master: 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581)
I20260811 22:11:44.269426 13672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:51579
I20260811 22:11:44.272300 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 14234
I20260811 22:11:44.272454 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:11:44.272877 10934 external_mini_cluster.cc:1428] Setting key 4c4f721943bad269fe7674e9b38270dd
I20260811 22:11:44.275883 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:11:44 dist-test-slave-68xk krb5kdc[13600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:11:44 dist-test-slave-68xk krb5kdc[13600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:44.330016 14266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.314357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33032 (local address 127.10.173.190:41811)
0811 22:11:44.314559 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.314564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.315124 (+ 560us) server_negotiation.cc:407] Connection header received
0811 22:11:44.315283 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.315288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.315361 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.315458 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:44.317106 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.317957 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.319432 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.320030 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.320150 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.324478 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:44.324484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:44.324487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:44.324526 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:44.327564 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:44.328285 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:44.328289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:44.328291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:44.328423 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:44.328779 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:44.328782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:44.328784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:44.329065 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:44.329273 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.329571 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.329748 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:11:44.334777 13672 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:11:44.342476 13672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:11:44.384302 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.359597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37200 (local address 127.10.173.130:35447)
0811 22:11:44.376515 (+ 16918us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.376524 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.376562 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:11:44.376645 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.376649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.376803 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.376982 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:44.377842 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.379354 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.381928 (+ 2574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.383213 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.383440 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.383637 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.383860 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.384020 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16695,"thread_start_us":16544,"threads_started":1}
I20260811 22:11:44.384284 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.356648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40210 (local address 127.10.173.129:38841)
0811 22:11:44.375103 (+ 18455us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.375114 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.375145 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:11:44.375262 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.375269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.375545 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.375796 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:44.376538 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.378214 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.381190 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.382759 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.382947 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.383144 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.383780 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.383958 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18270,"thread_start_us":18112,"threads_started":1}
I20260811 22:11:44.384358 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.359299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:35447 (local address 127.0.0.1:37200)
0811 22:11:44.373828 (+ 14529us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:44.373846 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:44.373957 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:44.376974 (+ 3017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:44.376979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:44.377008 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:44.377463 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.377478 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.379609 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.379615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.381741 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.381756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.383457 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.383463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.383615 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.383647 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:11:44.383743 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:11:44.383942 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14388,"mutex_wait_us":23,"thread_start_us":13946,"threads_started":1}
I20260811 22:11:44.385538 14462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.356652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:41581 (local address 127.0.0.1:49994)
0811 22:11:44.373015 (+ 16363us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:44.373036 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:44.373165 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:44.378278 (+ 5113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:44.378282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:44.378311 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:44.378803 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.378817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.380637 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.380646 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.382778 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.382793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.384704 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.384710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.384912 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.384946 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:11:44.385056 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 22:11:44.385198 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16228,"thread_start_us":9084,"threads_started":1}
I20260811 22:11:44.385936 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.356404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:38841 (local address 127.0.0.1:40210)
0811 22:11:44.366144 (+ 9740us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:44.366193 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:44.366376 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:44.375815 (+ 9439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:44.375822 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:44.375868 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:44.376280 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.376294 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.378476 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.378482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.381012 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.381027 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.383283 (+ 2256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.383289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.383521 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.383584 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 22:11:44.383717 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 22:11:44.385127 (+ 1410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9485,"mutex_wait_us":3,"thread_start_us":17773,"threads_started":1}
I20260811 22:11:44.386140 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.357119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49994 (local address 127.10.173.131:41581)
0811 22:11:44.377708 (+ 20589us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.377714 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.377737 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:11:44.377846 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.377852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.378065 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.379018 (+ 953us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:44.379302 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.380484 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.382960 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.384439 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.384710 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.384866 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.385145 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.385320 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20408,"thread_start_us":20200,"threads_started":1}
I20260811 22:11:44.389060 14110 tablet_service.cc:1467] Processing CreateTablet for tablet bcc4f1e8c2e147b2a17c14c68b21ad6f (DEFAULT_TABLE table=test-table [id=e64bef05993046e4bc0c0859ee25689e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:44.389292 14337 tablet_service.cc:1467] Processing CreateTablet for tablet bcc4f1e8c2e147b2a17c14c68b21ad6f (DEFAULT_TABLE table=test-table [id=e64bef05993046e4bc0c0859ee25689e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:44.389595 13891 tablet_service.cc:1467] Processing CreateTablet for tablet bcc4f1e8c2e147b2a17c14c68b21ad6f (DEFAULT_TABLE table=test-table [id=e64bef05993046e4bc0c0859ee25689e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:44.390601 14337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcc4f1e8c2e147b2a17c14c68b21ad6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:44.390640 14110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcc4f1e8c2e147b2a17c14c68b21ad6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:44.390929 13891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcc4f1e8c2e147b2a17c14c68b21ad6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:44.402184 14464 tablet_bootstrap.cc:492] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: Bootstrap starting.
I20260811 22:11:44.405441 14467 tablet_bootstrap.cc:492] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: Bootstrap starting.
I20260811 22:11:44.407143 14464 tablet_bootstrap.cc:654] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:44.408483 14464 log.cc:824] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:44.409178 14467 tablet_bootstrap.cc:654] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:44.409991 14466 tablet_bootstrap.cc:492] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: Bootstrap starting.
I20260811 22:11:44.410472 14467 log.cc:824] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:44.412182 14464 tablet_bootstrap.cc:492] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: No bootstrap required, opened a new log
I20260811 22:11:44.412508 14464 ts_tablet_manager.cc:1397] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 22:11:44.413651 14467 tablet_bootstrap.cc:492] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: No bootstrap required, opened a new log
I20260811 22:11:44.413965 14467 ts_tablet_manager.cc:1397] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 22:11:44.414155 14466 tablet_bootstrap.cc:654] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:44.415410 14466 log.cc:824] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:44.418324 14466 tablet_bootstrap.cc:492] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: No bootstrap required, opened a new log
I20260811 22:11:44.418643 14466 ts_tablet_manager.cc:1397] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 22:11:44.421512 14464 raft_consensus.cc:348] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } }
I20260811 22:11:44.421877 14464 raft_consensus.cc:374] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:44.421983 14464 raft_consensus.cc:729] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a482b2bfbc401eaf888d9fc560eab3, State: Initialized, Role: FOLLOWER
I20260811 22:11:44.422649 14464 consensus_queue.cc:260] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } }
I20260811 22:11:44.424239 14441 heartbeater.cc:503] Master 127.10.173.190:41811 was elected leader, sending a full tablet report...
I20260811 22:11:44.424852 14464 ts_tablet_manager.cc:1428] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260811 22:11:44.426316 14467 raft_consensus.cc:348] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } }
I20260811 22:11:44.426779 14467 raft_consensus.cc:374] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:44.426878 14467 raft_consensus.cc:729] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cea309ae45494e8b9f0ccd8319b47b, State: Initialized, Role: FOLLOWER
I20260811 22:11:44.427675 14467 consensus_queue.cc:260] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } }
I20260811 22:11:44.429194 14466 raft_consensus.cc:348] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } }
I20260811 22:11:44.429687 14466 raft_consensus.cc:374] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:44.429805 14466 raft_consensus.cc:729] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 722b01b4abad48a9ab3f2920c74be774, State: Initialized, Role: FOLLOWER
I20260811 22:11:44.430778 14466 consensus_queue.cc:260] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } }
I20260811 22:11:44.433998 14221 heartbeater.cc:503] Master 127.10.173.190:41811 was elected leader, sending a full tablet report...
I20260811 22:11:44.434310 14466 ts_tablet_manager.cc:1428] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260811 22:11:44.434672 14467 ts_tablet_manager.cc:1428] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: Time spent starting tablet: real 0.021s user 0.014s sys 0.004s
W20260811 22:11:44.446403 14226 tablet.cc:2369] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:11:44.515237 14442 tablet.cc:2369] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:11:44.535285 14480 raft_consensus.cc:482] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:44.535706 14480 raft_consensus.cc:504] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } }
I20260811 22:11:44.537537 14480 leader_election.cc:288] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581), 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447)
I20260811 22:11:44.544272 14477 raft_consensus.cc:482] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:44.544709 14477 raft_consensus.cc:504] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } }
I20260811 22:11:44.545325 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.538125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:52629 (local address 127.10.173.131:41581)
0811 22:11:44.538385 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.538396 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.539200 (+ 804us) server_negotiation.cc:407] Connection header received
0811 22:11:44.539556 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.539564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.539664 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.539743 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:44.540693 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.541390 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.543399 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.544325 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.544527 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.544615 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.544908 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.545059 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:11:44.545439 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.538003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:41581 (local address 127.10.173.129:52629)
0811 22:11:44.539075 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:44.539100 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:44.539344 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:44.540007 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:44.540012 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:44.540069 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:44.540540 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.540552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.541530 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.541535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.543234 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.543250 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.544554 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.544560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.544716 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.544744 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:11:44.544873 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 22:11:44.545001 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":922,"mutex_wait_us":97,"thread_start_us":737,"threads_started":1}
I20260811 22:11:44.545665 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.538166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:35241 (local address 127.10.173.130:35447)
0811 22:11:44.538401 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.538446 (+ 45us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.539279 (+ 833us) server_negotiation.cc:407] Connection header received
0811 22:11:44.539589 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.539610 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.539694 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.539806 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:44.540553 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.541382 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.543440 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.544355 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.544539 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.544613 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.545229 (+ 616us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.545372 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:11:44.546391 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.538065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:35447 (local address 127.10.173.129:35241)
0811 22:11:44.539124 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:44.539162 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:44.539376 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:44.539838 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:44.539843 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:44.539899 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:44.540364 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.540378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.541506 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.541514 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.543291 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.543303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.544805 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.544811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.545004 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.545052 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 22:11:44.545854 (+ 802us) client_negotiation.cc:240] Negotiation successful
0811 22:11:44.545999 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":34,"spinlock_wait_cycles":34048,"thread_start_us":599,"threads_started":1}
I20260811 22:11:44.546474 14375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc4f1e8c2e147b2a17c14c68b21ad6f" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a482b2bfbc401eaf888d9fc560eab3" is_pre_election: true
I20260811 22:11:44.546939 14375 raft_consensus.cc:2454] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 0.
I20260811 22:11:44.547690 13810 leader_election.cc:302] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a482b2bfbc401eaf888d9fc560eab3, 722b01b4abad48a9ab3f2920c74be774; no voters:
I20260811 22:11:44.548342 14155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc4f1e8c2e147b2a17c14c68b21ad6f" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b" is_pre_election: true
I20260811 22:11:44.548574 14480 raft_consensus.cc:2781] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:44.548728 14480 raft_consensus.cc:482] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:44.548851 14480 raft_consensus.cc:3037] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:44.548904 14155 raft_consensus.cc:2454] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 0.
I20260811 22:11:44.551474 14477 leader_election.cc:288] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841), 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447)
I20260811 22:11:44.553400 14480 raft_consensus.cc:504] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } }
I20260811 22:11:44.554211 14480 leader_election.cc:288] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 election: Requested vote from peers 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581), 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447)
I20260811 22:11:44.555007 14155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc4f1e8c2e147b2a17c14c68b21ad6f" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b"
I20260811 22:11:44.555225 14155 raft_consensus.cc:3037] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:44.556550 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.548303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:43765 (local address 127.10.173.129:38841)
0811 22:11:44.548516 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.548521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.548553 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:11:44.548621 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.548629 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.548699 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.548793 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:44.549654 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.551211 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.553804 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.555156 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.555317 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.555398 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.556191 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.556326 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:11:44.556872 14267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.547952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:38841 (local address 127.10.173.131:43765)
0811 22:11:44.548210 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:44.548229 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:44.548351 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:44.549169 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:44.549178 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:44.549220 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:44.549424 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.549440 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.551402 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.551407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.553591 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.553607 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.555496 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.555502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.555730 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.555763 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:11:44.555866 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:11:44.555997 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 22:11:44.558766 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.548286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:55679 (local address 127.10.173.130:35447)
0811 22:11:44.548773 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.548779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.552262 (+ 3483us) server_negotiation.cc:407] Connection header received
0811 22:11:44.552430 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.552434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.552488 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.552558 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:44.553222 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.554318 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.556692 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.557766 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.557870 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.557915 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.558377 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.558525 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362}
I20260811 22:11:44.559144 14375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc4f1e8c2e147b2a17c14c68b21ad6f" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a482b2bfbc401eaf888d9fc560eab3"
I20260811 22:11:44.559109 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc4f1e8c2e147b2a17c14c68b21ad6f" candidate_uuid: "27a482b2bfbc401eaf888d9fc560eab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722b01b4abad48a9ab3f2920c74be774" is_pre_election: true
I20260811 22:11:44.559366 14375 raft_consensus.cc:3037] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:44.559739 13929 raft_consensus.cc:2379] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 27a482b2bfbc401eaf888d9fc560eab3 in current term 1: Already voted for candidate 722b01b4abad48a9ab3f2920c74be774 in this term.
I20260811 22:11:44.560006 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.547962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:35447 (local address 127.10.173.131:55679)
0811 22:11:44.552152 (+ 4190us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:44.552170 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:44.552304 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:44.552596 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:44.552600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:44.552627 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:44.553033 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.553045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.554525 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.554530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.556479 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.556494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.558064 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.558068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.558191 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.558211 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:11:44.558949 (+ 738us) client_negotiation.cc:240] Negotiation successful
0811 22:11:44.559602 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4062,"mutex_wait_us":390,"thread_start_us":3456,"threads_started":1}
I20260811 22:11:44.560902 14154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcc4f1e8c2e147b2a17c14c68b21ad6f" candidate_uuid: "27a482b2bfbc401eaf888d9fc560eab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b" is_pre_election: true
I20260811 22:11:44.561878 14258 leader_election.cc:302] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 27a482b2bfbc401eaf888d9fc560eab3; no voters: 17cea309ae45494e8b9f0ccd8319b47b, 722b01b4abad48a9ab3f2920c74be774
I20260811 22:11:44.649753 14375 raft_consensus.cc:2454] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 1.
I20260811 22:11:44.649753 14155 raft_consensus.cc:2454] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 1.
W20260811 22:11:44.649881 14004 tablet.cc:2369] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:11:44.650307 14477 raft_consensus.cc:2726] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 22:11:44.650695 13817 leader_election.cc:302] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cea309ae45494e8b9f0ccd8319b47b, 722b01b4abad48a9ab3f2920c74be774; no voters:
I20260811 22:11:44.651124 14480 raft_consensus.cc:2781] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:44.651763 14480 raft_consensus.cc:686] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 1 LEADER]: Becoming Leader. State: Replica: 722b01b4abad48a9ab3f2920c74be774, State: Running, Role: LEADER
I20260811 22:11:44.652350 14480 consensus_queue.cc:237] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } }
I20260811 22:11:44.657719 13671 catalog_manager.cc:5495] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 reported cstate change: term changed from 0 to 1, leader changed from <none> to 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129). New cstate: current_term: 1 leader_uuid: "722b01b4abad48a9ab3f2920c74be774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:11:44 dist-test-slave-68xk krb5kdc[13600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486302, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:44.720696 14266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.705336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.10.173.190:41811)
0811 22:11:44.705575 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:44.705580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:44.706286 (+ 706us) server_negotiation.cc:407] Connection header received
0811 22:11:44.706404 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:44.706470 (+ 66us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:44.706550 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:44.706644 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:44.707965 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.708812 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.710167 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.710757 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.710880 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.714714 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:44.714719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:44.714723 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:44.714761 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:44.717527 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:44.718331 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:44.718335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:44.718337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:44.718535 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:44.718967 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:44.718974 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:44.718977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:44.719336 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:44.719546 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:44.719997 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 22:11:44.720265 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 22:11:44.720839 14500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:44.705361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41811 (local address 127.0.0.1:33048)
0811 22:11:44.706099 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:44.706147 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:44.706287 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:44.706728 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:44.706732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:44.707501 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:44.707772 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.707782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.708976 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.708981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.709965 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:44.709977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:44.710980 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:44.710985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:44.711142 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:44.711912 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:44.711929 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:44.714541 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:44.717690 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:44.717701 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:44.717708 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:44.718195 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:44.718693 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:44.718698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:44.718702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:44.718851 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:44.719536 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:44.719545 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:44.719739 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 22:11:44.719919 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 22:11:44.720400 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":197,"thread_start_us":419,"threads_started":1}
I20260811 22:11:44.727129 13671 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:33048:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:11:44.728457 13671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:11:44.735770 13891 tablet_service.cc:1467] Processing CreateTablet for tablet f99954ed41024dcba15974c38a2febd3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f3b2faef19646f4ae31d1f7dc282ef8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:11:44.736145 14110 tablet_service.cc:1467] Processing CreateTablet for tablet f99954ed41024dcba15974c38a2febd3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f3b2faef19646f4ae31d1f7dc282ef8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:11:44.736297 14337 tablet_service.cc:1467] Processing CreateTablet for tablet f99954ed41024dcba15974c38a2febd3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9f3b2faef19646f4ae31d1f7dc282ef8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:11:44.736796 13891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f99954ed41024dcba15974c38a2febd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:44.737146 14337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f99954ed41024dcba15974c38a2febd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:44.737139 14110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f99954ed41024dcba15974c38a2febd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:44.743042 14466 tablet_bootstrap.cc:492] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: Bootstrap starting.
I20260811 22:11:44.744486 14467 tablet_bootstrap.cc:492] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: Bootstrap starting.
I20260811 22:11:44.744486 14464 tablet_bootstrap.cc:492] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: Bootstrap starting.
I20260811 22:11:44.744580 14466 tablet_bootstrap.cc:654] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:44.746428 14467 tablet_bootstrap.cc:654] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:44.746433 14464 tablet_bootstrap.cc:654] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:44.747764 14466 tablet_bootstrap.cc:492] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: No bootstrap required, opened a new log
I20260811 22:11:44.747931 14466 ts_tablet_manager.cc:1397] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 22:11:44.749215 14467 tablet_bootstrap.cc:492] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: No bootstrap required, opened a new log
I20260811 22:11:44.749476 14467 ts_tablet_manager.cc:1397] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:11:44.748898 14466 raft_consensus.cc:348] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:44.749920 14466 raft_consensus.cc:374] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:44.749933 14464 tablet_bootstrap.cc:492] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: No bootstrap required, opened a new log
I20260811 22:11:44.750017 14466 raft_consensus.cc:729] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 722b01b4abad48a9ab3f2920c74be774, State: Initialized, Role: FOLLOWER
I20260811 22:11:44.750084 14464 ts_tablet_manager.cc:1397] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 22:11:44.750262 14466 consensus_queue.cc:260] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:44.750705 14480 tablet_replica.cc:439] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } }
I20260811 22:11:44.750988 14480 tablet_replica.cc:442] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:44.751448 14466 ts_tablet_manager.cc:1428] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 22:11:44.750864 14467 raft_consensus.cc:348] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:44.751724 14467 raft_consensus.cc:374] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:44.751828 14467 raft_consensus.cc:729] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cea309ae45494e8b9f0ccd8319b47b, State: Initialized, Role: FOLLOWER
I20260811 22:11:44.751681 14464 raft_consensus.cc:348] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:44.751936 14464 raft_consensus.cc:374] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:44.752035 14464 raft_consensus.cc:729] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a482b2bfbc401eaf888d9fc560eab3, State: Initialized, Role: FOLLOWER
I20260811 22:11:44.752156 14467 consensus_queue.cc:260] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:44.752314 14464 consensus_queue.cc:260] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:44.752662 14479 tablet_replica.cc:439] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } }
I20260811 22:11:44.753069 14479 tablet_replica.cc:442] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:44.753115 14477 tablet_replica.cc:439] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } }
I20260811 22:11:44.754546 14477 tablet_replica.cc:442] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:44.754892 14464 ts_tablet_manager.cc:1428] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:11:44.755959 14467 ts_tablet_manager.cc:1428] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 22:11:44.983400 14480 raft_consensus.cc:482] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:44.983620 14480 raft_consensus.cc:504] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:44.984270 14480 leader_election.cc:288] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447), 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581)
I20260811 22:11:44.984929 14375 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99954ed41024dcba15974c38a2febd3" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a482b2bfbc401eaf888d9fc560eab3" is_pre_election: true
I20260811 22:11:44.984938 14155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99954ed41024dcba15974c38a2febd3" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b" is_pre_election: true
I20260811 22:11:44.985234 14375 raft_consensus.cc:2454] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 0.
I20260811 22:11:44.985245 14155 raft_consensus.cc:2454] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 0.
I20260811 22:11:44.985723 13810 leader_election.cc:302] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a482b2bfbc401eaf888d9fc560eab3, 722b01b4abad48a9ab3f2920c74be774; no voters:
I20260811 22:11:44.986028 14480 raft_consensus.cc:2781] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:44.986135 14480 raft_consensus.cc:482] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:44.986219 14480 raft_consensus.cc:3037] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:45.041126 14487 consensus_queue.cc:1035] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:45.048101 14480 raft_consensus.cc:504] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.049000 14480 leader_election.cc:288] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 election: Requested vote from peers 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447), 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581)
I20260811 22:11:45.052228 14154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99954ed41024dcba15974c38a2febd3" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b"
I20260811 22:11:45.052538 14154 raft_consensus.cc:3037] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:45.054836 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f99954ed41024dcba15974c38a2febd3" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a482b2bfbc401eaf888d9fc560eab3"
I20260811 22:11:45.054836 14512 consensus_queue.cc:1035] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:45.055054 14374 raft_consensus.cc:3037] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:45.150266 14374 raft_consensus.cc:2454] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 1.
I20260811 22:11:45.150270 14154 raft_consensus.cc:2454] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 1.
I20260811 22:11:45.150941 13810 leader_election.cc:302] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a482b2bfbc401eaf888d9fc560eab3, 722b01b4abad48a9ab3f2920c74be774; no voters:
I20260811 22:11:45.151415 14512 raft_consensus.cc:2781] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:45.151598 14512 raft_consensus.cc:686] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [term 1 LEADER]: Becoming Leader. State: Replica: 722b01b4abad48a9ab3f2920c74be774, State: Running, Role: LEADER
I20260811 22:11:45.151873 14512 consensus_queue.cc:237] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.154140 14509 tablet_replica.cc:439] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: TxnStatusTablet state changed. Reason: New leader 722b01b4abad48a9ab3f2920c74be774. Latest consensus state: current_term: 1 leader_uuid: "722b01b4abad48a9ab3f2920c74be774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } }
I20260811 22:11:45.154471 14509 tablet_replica.cc:442] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:11:45.155747 14519 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:11:45.156004 14519 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:11:45.155591 13671 catalog_manager.cc:5495] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 reported cstate change: term changed from 0 to 1, leader changed from <none> to 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129). New cstate: current_term: 1 leader_uuid: "722b01b4abad48a9ab3f2920c74be774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260811 22:11:45.208279 14500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:45.201406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:38841 (local address 127.0.0.1:40222)
0811 22:11:45.201648 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:45.201692 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:45.201866 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:45.202736 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:45.202741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:45.202755 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:11:45.202999 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:45.203010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:45.204269 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:45.204277 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:45.205422 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:45.205435 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:45.206300 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:45.206307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:45.206572 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:45.206642 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:11:45.207634 (+ 992us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:11:45.207659 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:11:45.207751 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:11:45.207918 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:11:45.208369 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:45.201478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40222 (local address 127.10.173.129:38841)
0811 22:11:45.202272 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:45.202276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:45.202297 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:11:45.202385 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:45.202389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:45.202537 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:45.202711 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:45.203219 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:45.204118 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:45.205593 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:45.206124 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:45.206282 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:45.206889 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:45.207487 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:45.207672 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:45.207799 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 22:11:45.207891 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":358,"threads_started":1}
I20260811 22:11:45.215956 14154 raft_consensus.cc:1261] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Refusing update from remote peer 722b01b4abad48a9ab3f2920c74be774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:45.216048 14374 raft_consensus.cc:1261] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Refusing update from remote peer 722b01b4abad48a9ab3f2920c74be774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:45.216699 14509 consensus_queue.cc:1035] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:45.217057 14512 consensus_queue.cc:1035] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:45.224273 14479 tablet_replica.cc:439] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: TxnStatusTablet state changed. Reason: New leader 722b01b4abad48a9ab3f2920c74be774. Latest consensus state: current_term: 1 leader_uuid: "722b01b4abad48a9ab3f2920c74be774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } }
I20260811 22:11:45.224757 14479 tablet_replica.cc:442] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:45.228968 14515 mvcc.cc:205] Tried to move back new op lower bound from 7317447906156122112 to 7317447905904803840. Current Snapshot: MvccSnapshot[applied={T|T < 7317447906156122112}]
I20260811 22:11:45.229282 14510 mvcc.cc:205] Tried to move back new op lower bound from 7317447906156122112 to 7317447905904803840. Current Snapshot: MvccSnapshot[applied={T|T < 7317447906156122112}]
I20260811 22:11:45.237538 14477 tablet_replica.cc:439] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: TxnStatusTablet state changed. Reason: New leader 722b01b4abad48a9ab3f2920c74be774. Latest consensus state: current_term: 1 leader_uuid: "722b01b4abad48a9ab3f2920c74be774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } }
I20260811 22:11:45.237943 14477 tablet_replica.cc:442] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:45.239970 14479 tablet_replica.cc:439] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "722b01b4abad48a9ab3f2920c74be774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } }
I20260811 22:11:45.240247 14479 tablet_replica.cc:442] T f99954ed41024dcba15974c38a2febd3 P 17cea309ae45494e8b9f0ccd8319b47b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:45.240808 14509 tablet_replica.cc:439] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "722b01b4abad48a9ab3f2920c74be774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } }
I20260811 22:11:45.241115 14509 tablet_replica.cc:442] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:11:45.244879 14509 tablet_replica.cc:439] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "722b01b4abad48a9ab3f2920c74be774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } }
I20260811 22:11:45.245165 14509 tablet_replica.cc:442] T f99954ed41024dcba15974c38a2febd3 P 722b01b4abad48a9ab3f2920c74be774: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:11:45.252198 14477 tablet_replica.cc:439] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "722b01b4abad48a9ab3f2920c74be774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } } }
I20260811 22:11:45.252470 14477 tablet_replica.cc:442] T f99954ed41024dcba15974c38a2febd3 P 27a482b2bfbc401eaf888d9fc560eab3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:11:45.261219 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:45.252988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.10.173.129:38841)
0811 22:11:45.253301 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:45.253308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:45.253330 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:11:45.253568 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:45.253575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:45.253667 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:45.253732 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:45.254481 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:45.255835 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:45.258117 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:45.259625 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:45.259785 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:45.260303 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:45.260416 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:45.260498 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:45.260706 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 22:11:45.260790 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260811 22:11:45.283655 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:45.276246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:55443 (local address 127.10.173.129:38841)
0811 22:11:45.276442 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:45.276449 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:45.276527 (+ 78us) server_negotiation.cc:407] Connection header received
0811 22:11:45.276918 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:45.276924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:45.277006 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:45.277125 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:45.277766 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:45.279131 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:45.281293 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:45.282024 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:45.282182 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:45.282732 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:45.282898 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:45.282991 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:45.283167 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 22:11:45.283252 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:11:45.283762 14536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:45.275731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:38841 (local address 127.10.173.129:55443)
0811 22:11:45.276435 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:45.276456 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:45.276691 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:45.277143 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:45.277153 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:45.277167 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:11:45.277574 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:45.277586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:45.279280 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:45.279285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:45.281135 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:45.281150 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:45.282212 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:45.282221 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:45.282444 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:45.282516 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:11:45.283013 (+ 497us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:11:45.283024 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 22:11:45.283131 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:11:45.283288 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":813,"threads_started":1}
I20260811 22:11:45.355293 13669 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:45.355597 13669 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:45.359345 13669 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44: Sending DeleteTablet for 3 replicas of tablet bcc4f1e8c2e147b2a17c14c68b21ad6f
I20260811 22:11:45.360487 14337 tablet_service.cc:1514] Processing DeleteTablet for tablet bcc4f1e8c2e147b2a17c14c68b21ad6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:45 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:49994
I20260811 22:11:45.360488 13889 tablet_service.cc:1514] Processing DeleteTablet for tablet bcc4f1e8c2e147b2a17c14c68b21ad6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:45 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:40210
I20260811 22:11:45.368036 14110 tablet_service.cc:1514] Processing DeleteTablet for tablet bcc4f1e8c2e147b2a17c14c68b21ad6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:45 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:37200
I20260811 22:11:45.368600 14539 tablet_replica.cc:330] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: stopping tablet replica
I20260811 22:11:45.369197 14539 raft_consensus.cc:2229] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:45.369705 14539 raft_consensus.cc:2258] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:45.370222 13669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:11:45.370980 13669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:11:45.373744 14539 ts_tablet_manager.cc:1905] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:45.377887 14542 tablet_replica.cc:330] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: stopping tablet replica
I20260811 22:11:45.378453 14542 raft_consensus.cc:2229] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:11:45.380407 14542 raft_consensus.cc:2258] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:45.381701 14543 tablet_replica.cc:330] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: stopping tablet replica
I20260811 22:11:45.382162 14543 raft_consensus.cc:2229] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:45.382531 14543 raft_consensus.cc:2258] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:45.383546 13889 tablet_service.cc:1467] Processing CreateTablet for tablet af9774ec65df49fd8086769f1c583089 (DEFAULT_TABLE table=test-table [id=e69dc1f626064c379f2562f13a4b0117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:45.383978 14337 tablet_service.cc:1467] Processing CreateTablet for tablet af9774ec65df49fd8086769f1c583089 (DEFAULT_TABLE table=test-table [id=e69dc1f626064c379f2562f13a4b0117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:45.384138 13889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9774ec65df49fd8086769f1c583089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:45.384447 14337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9774ec65df49fd8086769f1c583089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:45.383567 14110 tablet_service.cc:1467] Processing CreateTablet for tablet af9774ec65df49fd8086769f1c583089 (DEFAULT_TABLE table=test-table [id=e69dc1f626064c379f2562f13a4b0117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:45.388653 14110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af9774ec65df49fd8086769f1c583089. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:45.391206 14543 ts_tablet_manager.cc:1905] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:45.391744 14542 ts_tablet_manager.cc:1905] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:45.395743 14544 tablet_bootstrap.cc:492] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b: Bootstrap starting.
I20260811 22:11:45.396871 14545 tablet_bootstrap.cc:492] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774: Bootstrap starting.
I20260811 22:11:45.398207 14544 tablet_bootstrap.cc:654] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:45.399257 14545 tablet_bootstrap.cc:654] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:45.401376 14544 tablet_bootstrap.cc:492] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b: No bootstrap required, opened a new log
I20260811 22:11:45.401525 14544 ts_tablet_manager.cc:1397] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 22:11:45.401836 14545 tablet_bootstrap.cc:492] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774: No bootstrap required, opened a new log
I20260811 22:11:45.401989 14545 ts_tablet_manager.cc:1397] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:11:45.402379 14544 raft_consensus.cc:348] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.402688 14544 raft_consensus.cc:374] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:45.402793 14544 raft_consensus.cc:729] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cea309ae45494e8b9f0ccd8319b47b, State: Initialized, Role: FOLLOWER
I20260811 22:11:45.402908 14545 raft_consensus.cc:348] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.403177 14545 raft_consensus.cc:374] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:45.403064 14544 consensus_queue.cc:260] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.403293 14545 raft_consensus.cc:729] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 722b01b4abad48a9ab3f2920c74be774, State: Initialized, Role: FOLLOWER
I20260811 22:11:45.403544 14545 consensus_queue.cc:260] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.403723 14544 ts_tablet_manager.cc:1428] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:11:45.404642 14545 ts_tablet_manager.cc:1428] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 22:11:45.470948 14543 ts_tablet_manager.cc:1918] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:11:45.471252 14543 log.cc:1196] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/bcc4f1e8c2e147b2a17c14c68b21ad6f
I20260811 22:11:45.471899 14543 ts_tablet_manager.cc:1939] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 17cea309ae45494e8b9f0ccd8319b47b: Deleting consensus metadata
I20260811 22:11:45.472919 14548 tablet_bootstrap.cc:492] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3: Bootstrap starting.
I20260811 22:11:45.475574 14548 tablet_bootstrap.cc:654] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:45.479403 14548 tablet_bootstrap.cc:492] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3: No bootstrap required, opened a new log
I20260811 22:11:45.479605 14548 ts_tablet_manager.cc:1397] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:11:45.480334 14548 raft_consensus.cc:348] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.480578 14548 raft_consensus.cc:374] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:45.480686 14548 raft_consensus.cc:729] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a482b2bfbc401eaf888d9fc560eab3, State: Initialized, Role: FOLLOWER
I20260811 22:11:45.480975 14548 consensus_queue.cc:260] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.481707 14548 ts_tablet_manager.cc:1428] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:11:45.490334 14480 raft_consensus.cc:482] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:45.490602 14480 raft_consensus.cc:504] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.491349 14480 leader_election.cc:288] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447), 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581)
I20260811 22:11:45.491917 14155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9774ec65df49fd8086769f1c583089" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b" is_pre_election: true
I20260811 22:11:45.492015 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9774ec65df49fd8086769f1c583089" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a482b2bfbc401eaf888d9fc560eab3" is_pre_election: true
I20260811 22:11:45.492230 14155 raft_consensus.cc:2454] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 0.
I20260811 22:11:45.492259 14374 raft_consensus.cc:2454] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 0.
I20260811 22:11:45.492743 13810 leader_election.cc:302] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a482b2bfbc401eaf888d9fc560eab3, 722b01b4abad48a9ab3f2920c74be774; no voters:
I20260811 22:11:45.493093 14480 raft_consensus.cc:2781] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:45.493213 14480 raft_consensus.cc:482] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:45.493335 14480 raft_consensus.cc:3037] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:45.553460 13645 catalog_manager.cc:4841] TS 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447): tablet bcc4f1e8c2e147b2a17c14c68b21ad6f (table test-table [id=e64bef05993046e4bc0c0859ee25689e]) successfully deleted
I20260811 22:11:45.565653 14542 ts_tablet_manager.cc:1918] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:11:45.565640 14539 ts_tablet_manager.cc:1918] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:11:45.565902 14542 log.cc:1196] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/bcc4f1e8c2e147b2a17c14c68b21ad6f
I20260811 22:11:45.565908 14539 log.cc:1196] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/bcc4f1e8c2e147b2a17c14c68b21ad6f
I20260811 22:11:45.566601 14539 ts_tablet_manager.cc:1939] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 27a482b2bfbc401eaf888d9fc560eab3: Deleting consensus metadata
I20260811 22:11:45.566612 14542 ts_tablet_manager.cc:1939] T bcc4f1e8c2e147b2a17c14c68b21ad6f P 722b01b4abad48a9ab3f2920c74be774: Deleting consensus metadata
I20260811 22:11:45.567757 14480 raft_consensus.cc:504] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.569865 14155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9774ec65df49fd8086769f1c583089" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b"
I20260811 22:11:45.570078 13650 catalog_manager.cc:4841] TS 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841): tablet bcc4f1e8c2e147b2a17c14c68b21ad6f (table test-table [id=e64bef05993046e4bc0c0859ee25689e]) successfully deleted
I20260811 22:11:45.570148 14155 raft_consensus.cc:3037] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:45.570309 13643 catalog_manager.cc:4841] TS 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581): tablet bcc4f1e8c2e147b2a17c14c68b21ad6f (table test-table [id=e64bef05993046e4bc0c0859ee25689e]) successfully deleted
I20260811 22:11:45.570708 14480 leader_election.cc:288] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 election: Requested vote from peers 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447), 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581)
I20260811 22:11:45.571259 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af9774ec65df49fd8086769f1c583089" candidate_uuid: "722b01b4abad48a9ab3f2920c74be774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a482b2bfbc401eaf888d9fc560eab3"
I20260811 22:11:45.571480 14374 raft_consensus.cc:3037] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:45.660183 14155 raft_consensus.cc:2454] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 1.
I20260811 22:11:45.660200 14374 raft_consensus.cc:2454] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 722b01b4abad48a9ab3f2920c74be774 in term 1.
I20260811 22:11:45.661027 13810 leader_election.cc:302] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a482b2bfbc401eaf888d9fc560eab3, 722b01b4abad48a9ab3f2920c74be774; no voters:
I20260811 22:11:45.661504 14480 raft_consensus.cc:2781] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:45.661757 14480 raft_consensus.cc:686] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 1 LEADER]: Becoming Leader. State: Replica: 722b01b4abad48a9ab3f2920c74be774, State: Running, Role: LEADER
I20260811 22:11:45.662081 14480 consensus_queue.cc:237] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:45.665844 13671 catalog_manager.cc:5495] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 reported cstate change: term changed from 0 to 1, leader changed from <none> to 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129). New cstate: current_term: 1 leader_uuid: "722b01b4abad48a9ab3f2920c74be774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } health_report { overall_health: UNKNOWN } } }
I20260811 22:11:45.706733 14155 raft_consensus.cc:1261] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Refusing update from remote peer 722b01b4abad48a9ab3f2920c74be774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:45.707494 14512 consensus_queue.cc:1035] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:45.710134 14374 raft_consensus.cc:1261] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Refusing update from remote peer 722b01b4abad48a9ab3f2920c74be774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:45.710841 14512 consensus_queue.cc:1035] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:11:45.747743 14510 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:11:47.345854 13671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:47.346122 13671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:47.350930 13671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44: Sending DeleteTablet for 3 replicas of tablet af9774ec65df49fd8086769f1c583089
I20260811 22:11:47.351786 13889 tablet_service.cc:1514] Processing DeleteTablet for tablet af9774ec65df49fd8086769f1c583089 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:47 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:40210
I20260811 22:11:47.351830 14110 tablet_service.cc:1514] Processing DeleteTablet for tablet af9774ec65df49fd8086769f1c583089 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:47 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:37200
I20260811 22:11:47.351962 14337 tablet_service.cc:1514] Processing DeleteTablet for tablet af9774ec65df49fd8086769f1c583089 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:47 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:49994
I20260811 22:11:47.362885 14592 tablet_replica.cc:330] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774: stopping tablet replica
I20260811 22:11:47.363158 14590 tablet_replica.cc:330] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b: stopping tablet replica
I20260811 22:11:47.363185 14592 raft_consensus.cc:2229] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:11:47.363435 14590 raft_consensus.cc:2229] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:47.363750 14592 raft_consensus.cc:2258] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:47.363732 14590 raft_consensus.cc:2258] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:47.364928 14592 ts_tablet_manager.cc:1905] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:47.365024 14590 ts_tablet_manager.cc:1905] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:47.370883 14593 tablet_replica.cc:330] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3: stopping tablet replica
I20260811 22:11:47.371124 14593 raft_consensus.cc:2229] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:47.371506 14593 raft_consensus.cc:2258] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:47.372130 14590 ts_tablet_manager.cc:1918] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:11:47.372359 14590 log.cc:1196] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/af9774ec65df49fd8086769f1c583089
I20260811 22:11:47.372733 14593 ts_tablet_manager.cc:1905] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:47.373023 14590 ts_tablet_manager.cc:1939] T af9774ec65df49fd8086769f1c583089 P 17cea309ae45494e8b9f0ccd8319b47b: Deleting consensus metadata
I20260811 22:11:47.373014 14592 ts_tablet_manager.cc:1918] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:11:47.373168 14592 log.cc:1196] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/af9774ec65df49fd8086769f1c583089
I20260811 22:11:47.373852 14592 ts_tablet_manager.cc:1939] T af9774ec65df49fd8086769f1c583089 P 722b01b4abad48a9ab3f2920c74be774: Deleting consensus metadata
I20260811 22:11:47.374473 13645 catalog_manager.cc:4841] TS 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447): tablet af9774ec65df49fd8086769f1c583089 (table test-table [id=e69dc1f626064c379f2562f13a4b0117]) successfully deleted
I20260811 22:11:47.375170 13650 catalog_manager.cc:4841] TS 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841): tablet af9774ec65df49fd8086769f1c583089 (table test-table [id=e69dc1f626064c379f2562f13a4b0117]) successfully deleted
I20260811 22:11:47.378684 14593 ts_tablet_manager.cc:1918] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:11:47.378857 14593 log.cc:1196] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/af9774ec65df49fd8086769f1c583089
I20260811 22:11:47.379271 14593 ts_tablet_manager.cc:1939] T af9774ec65df49fd8086769f1c583089 P 27a482b2bfbc401eaf888d9fc560eab3: Deleting consensus metadata
I20260811 22:11:47.380198 13643 catalog_manager.cc:4841] TS 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581): tablet af9774ec65df49fd8086769f1c583089 (table test-table [id=e69dc1f626064c379f2562f13a4b0117]) successfully deleted
Aug 11 22:11:47 dist-test-slave-68xk krb5kdc[13600](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:11:47 dist-test-slave-68xk krb5kdc[13600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:47.427071 14601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:47.406851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33054 (local address 127.10.173.190:41811)
0811 22:11:47.407508 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:47.407513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:47.407538 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:11:47.407788 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:47.407792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:47.407861 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:47.407965 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:47.409252 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.410099 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.411426 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.411890 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.411988 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:47.421605 (+ 9617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:47.421610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:47.421614 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:47.421651 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:47.424434 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:47.425074 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:47.425078 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:47.425081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:47.425222 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:47.425560 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:47.425563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:47.425565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:47.425849 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:47.426057 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:47.426351 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 22:11:47.426617 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":335,"threads_started":1}
I20260811 22:11:47.430941 13671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:11:47.431648 13671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:11:47.438490 13889 tablet_service.cc:1467] Processing CreateTablet for tablet feef24731d5e4ce39d263db71ac81e5a (DEFAULT_TABLE table=test-table [id=ac3f896244fb44f68f83e928aac0591c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:47.438791 14110 tablet_service.cc:1467] Processing CreateTablet for tablet feef24731d5e4ce39d263db71ac81e5a (DEFAULT_TABLE table=test-table [id=ac3f896244fb44f68f83e928aac0591c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:47.438985 13889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feef24731d5e4ce39d263db71ac81e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:47.438925 14337 tablet_service.cc:1467] Processing CreateTablet for tablet feef24731d5e4ce39d263db71ac81e5a (DEFAULT_TABLE table=test-table [id=ac3f896244fb44f68f83e928aac0591c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:47.439348 14110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feef24731d5e4ce39d263db71ac81e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:47.439363 14337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feef24731d5e4ce39d263db71ac81e5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:47.445648 14602 tablet_bootstrap.cc:492] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774: Bootstrap starting.
I20260811 22:11:47.447507 14604 tablet_bootstrap.cc:492] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b: Bootstrap starting.
I20260811 22:11:47.447933 14602 tablet_bootstrap.cc:654] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:47.449816 14604 tablet_bootstrap.cc:654] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:47.451156 14602 tablet_bootstrap.cc:492] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774: No bootstrap required, opened a new log
I20260811 22:11:47.451356 14602 ts_tablet_manager.cc:1397] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:11:47.451479 14605 tablet_bootstrap.cc:492] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3: Bootstrap starting.
I20260811 22:11:47.452365 14602 raft_consensus.cc:348] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.452661 14602 raft_consensus.cc:374] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:47.452821 14602 raft_consensus.cc:729] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 722b01b4abad48a9ab3f2920c74be774, State: Initialized, Role: FOLLOWER
I20260811 22:11:47.453153 14602 consensus_queue.cc:260] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.453490 14605 tablet_bootstrap.cc:654] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:47.453847 14602 ts_tablet_manager.cc:1428] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 22:11:47.454766 14604 tablet_bootstrap.cc:492] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b: No bootstrap required, opened a new log
I20260811 22:11:47.454943 14604 ts_tablet_manager.cc:1397] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.004s
I20260811 22:11:47.456020 14604 raft_consensus.cc:348] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.456295 14604 raft_consensus.cc:374] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:47.456434 14604 raft_consensus.cc:729] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cea309ae45494e8b9f0ccd8319b47b, State: Initialized, Role: FOLLOWER
I20260811 22:11:47.456741 14604 consensus_queue.cc:260] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.457116 14605 tablet_bootstrap.cc:492] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3: No bootstrap required, opened a new log
I20260811 22:11:47.457314 14605 ts_tablet_manager.cc:1397] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260811 22:11:47.458945 14605 raft_consensus.cc:348] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.459293 14605 raft_consensus.cc:374] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:47.459445 14605 raft_consensus.cc:729] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a482b2bfbc401eaf888d9fc560eab3, State: Initialized, Role: FOLLOWER
I20260811 22:11:47.459803 14605 consensus_queue.cc:260] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.461014 14605 ts_tablet_manager.cc:1428] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 22:11:47.461838 14604 ts_tablet_manager.cc:1428] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260811 22:11:47.770828 14614 raft_consensus.cc:482] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:47.771178 14614 raft_consensus.cc:504] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.772045 14614 leader_election.cc:288] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841), 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447)
I20260811 22:11:47.772779 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feef24731d5e4ce39d263db71ac81e5a" candidate_uuid: "27a482b2bfbc401eaf888d9fc560eab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722b01b4abad48a9ab3f2920c74be774" is_pre_election: true
I20260811 22:11:47.772840 14154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feef24731d5e4ce39d263db71ac81e5a" candidate_uuid: "27a482b2bfbc401eaf888d9fc560eab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b" is_pre_election: true
I20260811 22:11:47.773067 13929 raft_consensus.cc:2454] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a482b2bfbc401eaf888d9fc560eab3 in term 0.
I20260811 22:11:47.773406 14154 raft_consensus.cc:2454] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a482b2bfbc401eaf888d9fc560eab3 in term 0.
I20260811 22:11:47.773738 14259 leader_election.cc:302] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a482b2bfbc401eaf888d9fc560eab3, 722b01b4abad48a9ab3f2920c74be774; no voters:
I20260811 22:11:47.774127 14614 raft_consensus.cc:2781] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:47.774293 14614 raft_consensus.cc:482] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:47.774451 14614 raft_consensus.cc:3037] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:47.776983 14614 raft_consensus.cc:504] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.777621 14614 leader_election.cc:288] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [CANDIDATE]: Term 1 election: Requested vote from peers 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841), 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447)
I20260811 22:11:47.778098 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feef24731d5e4ce39d263db71ac81e5a" candidate_uuid: "27a482b2bfbc401eaf888d9fc560eab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722b01b4abad48a9ab3f2920c74be774"
I20260811 22:11:47.778206 14154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feef24731d5e4ce39d263db71ac81e5a" candidate_uuid: "27a482b2bfbc401eaf888d9fc560eab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b"
I20260811 22:11:47.778317 13929 raft_consensus.cc:3037] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:47.778528 14154 raft_consensus.cc:3037] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:47.780637 13929 raft_consensus.cc:2454] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a482b2bfbc401eaf888d9fc560eab3 in term 1.
I20260811 22:11:47.780821 14154 raft_consensus.cc:2454] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a482b2bfbc401eaf888d9fc560eab3 in term 1.
I20260811 22:11:47.781219 14259 leader_election.cc:302] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27a482b2bfbc401eaf888d9fc560eab3, 722b01b4abad48a9ab3f2920c74be774; no voters:
I20260811 22:11:47.781534 14614 raft_consensus.cc:2781] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:47.782164 14614 raft_consensus.cc:686] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 LEADER]: Becoming Leader. State: Replica: 27a482b2bfbc401eaf888d9fc560eab3, State: Running, Role: LEADER
I20260811 22:11:47.782639 14614 consensus_queue.cc:237] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.785957 13669 catalog_manager.cc:5495] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131). New cstate: current_term: 1 leader_uuid: "27a482b2bfbc401eaf888d9fc560eab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260811 22:11:47.826807 14623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:47.820496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49996 (local address 127.10.173.131:41581)
0811 22:11:47.821209 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:47.821214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:47.821249 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:11:47.821334 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:47.821338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:47.821412 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:47.821585 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:47.821936 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.822906 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.824310 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.824781 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.824879 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:47.825368 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:47.825943 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:47.826061 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:47.826168 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 22:11:47.826235 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":387,"threads_started":1}
I20260811 22:11:47.842079 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:47.836608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:45053 (local address 127.10.173.129:38841)
0811 22:11:47.837190 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:47.837196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:47.837483 (+ 287us) server_negotiation.cc:407] Connection header received
0811 22:11:47.837625 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:47.837629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:47.837700 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:47.837784 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:47.838258 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.839073 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.840497 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.840923 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.841021 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:47.841461 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:47.841556 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:47.841625 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:47.841743 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 22:11:47.841807 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":299,"threads_started":1}
I20260811 22:11:47.842387 14628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:47.836568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:38841 (local address 127.10.173.131:45053)
0811 22:11:47.837380 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:47.837404 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:47.837519 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:47.837812 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:47.837815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:47.837824 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:11:47.838111 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:47.838118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.839191 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.839194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:47.840372 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:47.840384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.841082 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.841086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:47.841222 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:47.841275 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:11:47.841640 (+ 365us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:11:47.841648 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:11:47.841706 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 22:11:47.841806 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":521,"threads_started":1}
I20260811 22:11:47.850538 14154 raft_consensus.cc:1261] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Refusing update from remote peer 27a482b2bfbc401eaf888d9fc560eab3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:47.850884 13929 raft_consensus.cc:1261] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Refusing update from remote peer 27a482b2bfbc401eaf888d9fc560eab3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:47.851194 14617 consensus_queue.cc:1035] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:47.851858 14614 consensus_queue.cc:1035] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:47.880887 14623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:47.874273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:57979 (local address 127.10.173.131:41581)
0811 22:11:47.874512 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:47.874522 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:47.875189 (+ 667us) server_negotiation.cc:407] Connection header received
0811 22:11:47.875416 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:47.875421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:47.875505 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:47.875589 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:47.876319 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.877317 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.879010 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.879531 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.879682 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:47.880148 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:47.880270 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:47.880368 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:47.880503 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 22:11:47.880583 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:11:47.881165 14634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:47.874164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:41581 (local address 127.10.173.129:57979)
0811 22:11:47.875092 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:47.875113 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:47.875268 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:47.875788 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:47.875793 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:47.875807 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:11:47.876105 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:47.876115 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.877463 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.877467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:47.878879 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:47.878892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:47.879733 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:47.879741 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:47.879923 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:47.879961 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:11:47.880372 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:11:47.880385 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 22:11:47.880480 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:11:47.880618 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":567,"threads_started":1}
I20260811 22:11:47.917440 13671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:47.917776 13671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:47.921727 13671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44: Sending DeleteTablet for 3 replicas of tablet feef24731d5e4ce39d263db71ac81e5a
I20260811 22:11:47.922762 13889 tablet_service.cc:1514] Processing DeleteTablet for tablet feef24731d5e4ce39d263db71ac81e5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:47 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:40210
I20260811 22:11:47.922919 14337 tablet_service.cc:1514] Processing DeleteTablet for tablet feef24731d5e4ce39d263db71ac81e5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:47 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:49994
I20260811 22:11:47.923053 14110 tablet_service.cc:1514] Processing DeleteTablet for tablet feef24731d5e4ce39d263db71ac81e5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:47 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:37200
I20260811 22:11:47.924186 13671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33054:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:11:47.925113 13671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:11:47.935246 13891 tablet_service.cc:1467] Processing CreateTablet for tablet a1f449f3705a40359eb6d2098206c117 (DEFAULT_TABLE table=test-table [id=55f8d9eefdc541fc8f2133b0e332d203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:47.935624 14109 tablet_service.cc:1467] Processing CreateTablet for tablet a1f449f3705a40359eb6d2098206c117 (DEFAULT_TABLE table=test-table [id=55f8d9eefdc541fc8f2133b0e332d203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:47.935772 13891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f449f3705a40359eb6d2098206c117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:47.936100 14109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f449f3705a40359eb6d2098206c117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:47.936271 14336 tablet_service.cc:1467] Processing CreateTablet for tablet a1f449f3705a40359eb6d2098206c117 (DEFAULT_TABLE table=test-table [id=55f8d9eefdc541fc8f2133b0e332d203]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:47.936913 14336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f449f3705a40359eb6d2098206c117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:47.942934 14602 tablet_bootstrap.cc:492] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774: Bootstrap starting.
I20260811 22:11:47.943243 14604 tablet_bootstrap.cc:492] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b: Bootstrap starting.
I20260811 22:11:47.944871 14641 tablet_replica.cc:330] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b: stopping tablet replica
I20260811 22:11:47.945109 14641 raft_consensus.cc:2229] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:47.945396 14602 tablet_bootstrap.cc:654] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:47.945430 14604 tablet_bootstrap.cc:654] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:47.945575 14641 raft_consensus.cc:2258] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:47.948274 14641 ts_tablet_manager.cc:1905] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:47.954594 14638 tablet_replica.cc:330] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774: stopping tablet replica
I20260811 22:11:47.954900 14638 raft_consensus.cc:2229] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:47.957350 14602 tablet_bootstrap.cc:492] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774: No bootstrap required, opened a new log
I20260811 22:11:47.957861 14602 ts_tablet_manager.cc:1397] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.009s
I20260811 22:11:47.959079 14602 raft_consensus.cc:348] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.959422 14602 raft_consensus.cc:374] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:47.959530 14602 raft_consensus.cc:729] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 722b01b4abad48a9ab3f2920c74be774, State: Initialized, Role: FOLLOWER
I20260811 22:11:47.959877 14602 consensus_queue.cc:260] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.960582 14602 ts_tablet_manager.cc:1428] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 22:11:47.960687 14638 raft_consensus.cc:2258] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:47.962034 14638 ts_tablet_manager.cc:1905] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:47.964980 14639 tablet_replica.cc:330] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3: stopping tablet replica
I20260811 22:11:47.965227 14639 raft_consensus.cc:2229] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:11:47.966032 14639 raft_consensus.cc:2258] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:47.967080 14639 ts_tablet_manager.cc:1905] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:47.967355 14645 tablet_bootstrap.cc:492] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3: Bootstrap starting.
I20260811 22:11:47.968070 14641 ts_tablet_manager.cc:1918] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:11:47.968314 14641 log.cc:1196] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/feef24731d5e4ce39d263db71ac81e5a
I20260811 22:11:47.968919 14641 ts_tablet_manager.cc:1939] T feef24731d5e4ce39d263db71ac81e5a P 17cea309ae45494e8b9f0ccd8319b47b: Deleting consensus metadata
I20260811 22:11:47.969281 14645 tablet_bootstrap.cc:654] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:47.970324 13645 catalog_manager.cc:4841] TS 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447): tablet feef24731d5e4ce39d263db71ac81e5a (table test-table [id=ac3f896244fb44f68f83e928aac0591c]) successfully deleted
I20260811 22:11:47.971316 14638 ts_tablet_manager.cc:1918] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:11:47.971519 14638 log.cc:1196] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/feef24731d5e4ce39d263db71ac81e5a
I20260811 22:11:47.972095 14638 ts_tablet_manager.cc:1939] T feef24731d5e4ce39d263db71ac81e5a P 722b01b4abad48a9ab3f2920c74be774: Deleting consensus metadata
I20260811 22:11:47.973474 14645 tablet_bootstrap.cc:492] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3: No bootstrap required, opened a new log
I20260811 22:11:47.973641 14645 ts_tablet_manager.cc:1397] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:11:47.973945 14604 tablet_bootstrap.cc:492] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b: No bootstrap required, opened a new log
I20260811 22:11:47.974140 14604 ts_tablet_manager.cc:1397] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b: Time spent bootstrapping tablet: real 0.031s user 0.010s sys 0.006s
I20260811 22:11:47.974593 14645 raft_consensus.cc:348] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.974910 14645 raft_consensus.cc:374] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:47.975116 14639 ts_tablet_manager.cc:1918] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:11:47.975114 14604 raft_consensus.cc:348] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.975180 14645 raft_consensus.cc:729] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a482b2bfbc401eaf888d9fc560eab3, State: Initialized, Role: FOLLOWER
I20260811 22:11:47.975445 14604 raft_consensus.cc:374] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:47.975557 14604 raft_consensus.cc:729] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17cea309ae45494e8b9f0ccd8319b47b, State: Initialized, Role: FOLLOWER
I20260811 22:11:47.975598 14639 log.cc:1196] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/feef24731d5e4ce39d263db71ac81e5a
I20260811 22:11:47.975703 14645 consensus_queue.cc:260] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.975941 14604 consensus_queue.cc:260] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:47.976775 14639 ts_tablet_manager.cc:1939] T feef24731d5e4ce39d263db71ac81e5a P 27a482b2bfbc401eaf888d9fc560eab3: Deleting consensus metadata
I20260811 22:11:47.976719 14604 ts_tablet_manager.cc:1428] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 22:11:47.977312 13650 catalog_manager.cc:4841] TS 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841): tablet feef24731d5e4ce39d263db71ac81e5a (table test-table [id=ac3f896244fb44f68f83e928aac0591c]) successfully deleted
I20260811 22:11:47.979068 13643 catalog_manager.cc:4841] TS 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581): tablet feef24731d5e4ce39d263db71ac81e5a (table test-table [id=ac3f896244fb44f68f83e928aac0591c]) successfully deleted
I20260811 22:11:47.980755 14645 ts_tablet_manager.cc:1428] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 22:11:48.146113 14613 raft_consensus.cc:482] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:48.146440 14613 raft_consensus.cc:504] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:48.148403 14613 leader_election.cc:288] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841), 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581)
I20260811 22:11:48.169018 14636 raft_consensus.cc:482] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:48.169299 14636 raft_consensus.cc:504] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:48.170244 14636 leader_election.cc:288] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841), 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447)
I20260811 22:11:48.170822 14648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:48.148894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:41581 (local address 127.10.173.130:36829)
0811 22:11:48.162266 (+ 13372us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:48.162285 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:48.162399 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:48.162969 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:48.162974 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:48.163002 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:48.163457 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:48.163467 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:48.165018 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:48.165023 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:48.167376 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:48.167391 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:48.169549 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:48.169555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:48.169732 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:48.169764 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:11:48.170285 (+ 521us) client_negotiation.cc:240] Negotiation successful
0811 22:11:48.170458 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13252,"thread_start_us":12826,"threads_started":1}
I20260811 22:11:48.171406 14623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:48.149320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:36829 (local address 127.10.173.131:41581)
0811 22:11:48.149502 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:48.149508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:48.162387 (+ 12879us) server_negotiation.cc:407] Connection header received
0811 22:11:48.162596 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:48.162602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:48.162712 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:48.162780 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:48.163674 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:48.164831 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:48.167544 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:48.169124 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:48.169292 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:48.169363 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:48.169926 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 22:11:48.170080 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:11:48.171408 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f449f3705a40359eb6d2098206c117" candidate_uuid: "27a482b2bfbc401eaf888d9fc560eab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722b01b4abad48a9ab3f2920c74be774" is_pre_election: true
I20260811 22:11:48.171669 13929 raft_consensus.cc:2454] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a482b2bfbc401eaf888d9fc560eab3 in term 0.
I20260811 22:11:48.172030 14154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f449f3705a40359eb6d2098206c117" candidate_uuid: "27a482b2bfbc401eaf888d9fc560eab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b" is_pre_election: true
I20260811 22:11:48.172333 14154 raft_consensus.cc:2454] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27a482b2bfbc401eaf888d9fc560eab3 in term 0.
I20260811 22:11:48.172628 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f449f3705a40359eb6d2098206c117" candidate_uuid: "17cea309ae45494e8b9f0ccd8319b47b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a482b2bfbc401eaf888d9fc560eab3" is_pre_election: true
I20260811 22:11:48.172770 14627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:48.149136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:41975 (local address 127.10.173.129:38841)
0811 22:11:48.149350 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:48.149356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:48.161547 (+ 12191us) server_negotiation.cc:407] Connection header received
0811 22:11:48.161774 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:48.161779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:48.161857 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:48.161961 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:48.163166 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:48.164284 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:48.167115 (+ 2831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:48.168515 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:48.168665 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:48.168746 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:48.172401 (+ 3655us) server_negotiation.cc:299] Negotiation successful
0811 22:11:48.172540 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:11:48.172889 14374 raft_consensus.cc:2454] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17cea309ae45494e8b9f0ccd8319b47b in term 0.
I20260811 22:11:48.173254 14258 leader_election.cc:302] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cea309ae45494e8b9f0ccd8319b47b, 27a482b2bfbc401eaf888d9fc560eab3; no voters:
I20260811 22:11:48.173524 14649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:48.148728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:38841 (local address 127.10.173.130:41975)
0811 22:11:48.161428 (+ 12700us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:48.161449 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:48.161624 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:48.162521 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:48.162526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:48.162557 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:48.162954 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:48.162969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:48.164495 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:48.164502 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:48.166929 (+ 2427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:48.166946 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:48.171997 (+ 5051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:48.172003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:48.172220 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:48.172252 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:11:48.172994 (+ 742us) client_negotiation.cc:240] Negotiation successful
0811 22:11:48.173143 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12549,"thread_start_us":13327,"threads_started":1}
I20260811 22:11:48.174222 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f449f3705a40359eb6d2098206c117" candidate_uuid: "17cea309ae45494e8b9f0ccd8319b47b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722b01b4abad48a9ab3f2920c74be774" is_pre_election: true
I20260811 22:11:48.174315 14636 raft_consensus.cc:2781] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:48.174466 14636 raft_consensus.cc:482] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:48.174556 14636 raft_consensus.cc:3037] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:48.174513 13929 raft_consensus.cc:2454] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17cea309ae45494e8b9f0ccd8319b47b in term 0.
I20260811 22:11:48.175087 14035 leader_election.cc:302] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17cea309ae45494e8b9f0ccd8319b47b, 722b01b4abad48a9ab3f2920c74be774; no voters:
I20260811 22:11:48.175607 14613 raft_consensus.cc:2781] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:48.175719 14613 raft_consensus.cc:482] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:48.175834 14613 raft_consensus.cc:3037] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:48.178025 14636 raft_consensus.cc:504] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:48.179451 14636 leader_election.cc:288] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [CANDIDATE]: Term 1 election: Requested vote from peers 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841), 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447)
I20260811 22:11:48.180015 13929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f449f3705a40359eb6d2098206c117" candidate_uuid: "27a482b2bfbc401eaf888d9fc560eab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722b01b4abad48a9ab3f2920c74be774"
I20260811 22:11:48.179983 14154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f449f3705a40359eb6d2098206c117" candidate_uuid: "27a482b2bfbc401eaf888d9fc560eab3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17cea309ae45494e8b9f0ccd8319b47b"
I20260811 22:11:48.180202 13929 raft_consensus.cc:3037] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:48.180366 14613 raft_consensus.cc:504] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:48.180912 14154 raft_consensus.cc:2379] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 27a482b2bfbc401eaf888d9fc560eab3 in current term 1: Already voted for candidate 17cea309ae45494e8b9f0ccd8319b47b in this term.
I20260811 22:11:48.181213 14613 leader_election.cc:288] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [CANDIDATE]: Term 1 election: Requested vote from peers 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841), 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581)
I20260811 22:11:48.182137 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f449f3705a40359eb6d2098206c117" candidate_uuid: "17cea309ae45494e8b9f0ccd8319b47b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27a482b2bfbc401eaf888d9fc560eab3"
I20260811 22:11:48.182430 14374 raft_consensus.cc:2379] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 17cea309ae45494e8b9f0ccd8319b47b in current term 1: Already voted for candidate 27a482b2bfbc401eaf888d9fc560eab3 in this term.
I20260811 22:11:48.183198 13928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f449f3705a40359eb6d2098206c117" candidate_uuid: "17cea309ae45494e8b9f0ccd8319b47b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722b01b4abad48a9ab3f2920c74be774"
I20260811 22:11:48.183529 13929 raft_consensus.cc:2454] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27a482b2bfbc401eaf888d9fc560eab3 in term 1.
I20260811 22:11:48.184093 14259 leader_election.cc:302] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 27a482b2bfbc401eaf888d9fc560eab3, 722b01b4abad48a9ab3f2920c74be774; no voters: 17cea309ae45494e8b9f0ccd8319b47b
I20260811 22:11:48.184154 14035 leader_election.cc:302] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 17cea309ae45494e8b9f0ccd8319b47b; no voters: 27a482b2bfbc401eaf888d9fc560eab3, 722b01b4abad48a9ab3f2920c74be774
I20260811 22:11:48.184520 14636 raft_consensus.cc:2781] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:48.184676 14613 raft_consensus.cc:2726] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 22:11:48.184697 14636 raft_consensus.cc:686] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 LEADER]: Becoming Leader. State: Replica: 27a482b2bfbc401eaf888d9fc560eab3, State: Running, Role: LEADER
I20260811 22:11:48.185081 14636 consensus_queue.cc:237] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } }
I20260811 22:11:48.191974 13669 catalog_manager.cc:5495] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131). New cstate: current_term: 1 leader_uuid: "27a482b2bfbc401eaf888d9fc560eab3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27a482b2bfbc401eaf888d9fc560eab3" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 41581 } health_report { overall_health: HEALTHY } } }
I20260811 22:11:48.266325 13929 raft_consensus.cc:1261] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Refusing update from remote peer 27a482b2bfbc401eaf888d9fc560eab3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:48.266705 14155 raft_consensus.cc:1261] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Refusing update from remote peer 27a482b2bfbc401eaf888d9fc560eab3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:48.266992 14614 consensus_queue.cc:1035] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "722b01b4abad48a9ab3f2920c74be774" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:48.267541 14631 consensus_queue.cc:1035] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17cea309ae45494e8b9f0ccd8319b47b" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 35447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:11:48.291317 14562 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:11:49.885181 13672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:49.885502 13672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33054:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:49.889847 13672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0f1844ac4f14edeaa99b9e782aedf44: Sending DeleteTablet for 3 replicas of tablet a1f449f3705a40359eb6d2098206c117
I20260811 22:11:49.890666 13891 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f449f3705a40359eb6d2098206c117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:49 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:40210
I20260811 22:11:49.890900 14336 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f449f3705a40359eb6d2098206c117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:49 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:49994
I20260811 22:11:49.890937 14110 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f449f3705a40359eb6d2098206c117 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:49 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:37200
I20260811 22:11:49.891912 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13786
I20260811 22:11:49.902112 14690 tablet_replica.cc:330] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3: stopping tablet replica
I20260811 22:11:49.902405 14690 raft_consensus.cc:2229] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:11:49.902884 14690 raft_consensus.cc:2258] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:49.903641 14689 tablet_replica.cc:330] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774: stopping tablet replica
I20260811 22:11:49.903904 14689 raft_consensus.cc:2229] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:49.904138 14692 tablet_replica.cc:330] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b: stopping tablet replica
I20260811 22:11:49.904260 14689 raft_consensus.cc:2258] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:49.904405 14692 raft_consensus.cc:2229] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:49.904657 14692 raft_consensus.cc:2258] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:49.905573 14692 ts_tablet_manager.cc:1905] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:49.905674 14690 ts_tablet_manager.cc:1905] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:49.907084 14689 ts_tablet_manager.cc:1905] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:49.912919 14690 ts_tablet_manager.cc:1918] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:11:49.913106 14690 log.cc:1196] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/a1f449f3705a40359eb6d2098206c117
I20260811 22:11:49.913568 14690 ts_tablet_manager.cc:1939] T a1f449f3705a40359eb6d2098206c117 P 27a482b2bfbc401eaf888d9fc560eab3: Deleting consensus metadata
I20260811 22:11:49.913903 14692 ts_tablet_manager.cc:1918] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:11:49.914130 14692 log.cc:1196] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/a1f449f3705a40359eb6d2098206c117
I20260811 22:11:49.914697 14692 ts_tablet_manager.cc:1939] T a1f449f3705a40359eb6d2098206c117 P 17cea309ae45494e8b9f0ccd8319b47b: Deleting consensus metadata
I20260811 22:11:49.914856 13643 catalog_manager.cc:4841] TS 27a482b2bfbc401eaf888d9fc560eab3 (127.10.173.131:41581): tablet a1f449f3705a40359eb6d2098206c117 (table test-table [id=55f8d9eefdc541fc8f2133b0e332d203]) successfully deleted
I20260811 22:11:49.915154 14689 ts_tablet_manager.cc:1918] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:11:49.915304 14689 log.cc:1196] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/a1f449f3705a40359eb6d2098206c117
I20260811 22:11:49.915759 14689 ts_tablet_manager.cc:1939] T a1f449f3705a40359eb6d2098206c117 P 722b01b4abad48a9ab3f2920c74be774: Deleting consensus metadata
I20260811 22:11:49.915886 13645 catalog_manager.cc:4841] TS 17cea309ae45494e8b9f0ccd8319b47b (127.10.173.130:35447): tablet a1f449f3705a40359eb6d2098206c117 (table test-table [id=55f8d9eefdc541fc8f2133b0e332d203]) successfully deleted
I20260811 22:11:49.916817 13650 catalog_manager.cc:4841] TS 722b01b4abad48a9ab3f2920c74be774 (127.10.173.129:38841): tablet a1f449f3705a40359eb6d2098206c117 (table test-table [id=55f8d9eefdc541fc8f2133b0e332d203]) successfully deleted
Aug 11 22:11:49 dist-test-slave-68xk krb5kdc[13600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:11:49.927392 14694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:49.894147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40230 (local address 127.10.173.129:38841)
0811 22:11:49.911099 (+ 16952us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:49.911105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:49.911137 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:11:49.911203 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:49.911208 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:49.911297 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:49.911437 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:49.913211 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:49.914382 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:49.916320 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:49.916886 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:49.917041 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:49.921369 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:49.921384 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:49.921402 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:49.921441 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:49.924445 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:49.925072 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:49.925079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:49.925081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:49.925262 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:49.925605 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:49.925608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:49.925610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:49.926013 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:49.926251 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:49.926643 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 22:11:49.926882 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16801,"thread_start_us":17454,"threads_started":1}
W20260811 22:11:49.928123 13994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40230
W20260811 22:11:49.928905 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13786: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:11:49.929201 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 13786
I20260811 22:11:49.964879 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 14007
Aug 11 22:11:49 dist-test-slave-68xk krb5kdc[13600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:11:49.982731 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:49.965720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37204 (local address 127.10.173.130:35447)
0811 22:11:49.966351 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:49.966356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:49.966372 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:11:49.966480 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:49.966486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:49.966577 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:49.966745 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:49.968200 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:49.969026 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:49.970578 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:49.971060 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:49.971216 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:49.976241 (+ 5025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:49.976257 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:49.976274 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:49.976313 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:49.979723 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:49.980457 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:49.980466 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:49.980468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:49.980598 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:49.980974 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:49.980980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:49.980985 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:49.981383 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:49.981704 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:49.982018 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 22:11:49.982276 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":308,"threads_started":1}
W20260811 22:11:49.983563 14211 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37204
W20260811 22:11:49.984412 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 14007: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:11:49.984555 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 14007
I20260811 22:11:50.018005 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 14234
Aug 11 22:11:50 dist-test-slave-68xk krb5kdc[13600](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:11:50.039345 14697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:50.019007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.10.173.131:41581)
0811 22:11:50.020344 (+ 1337us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:50.020351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:50.020372 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:11:50.020436 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:50.020440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:50.020502 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:50.020648 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:50.022104 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:50.022952 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:50.024741 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:50.025477 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:50.025645 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:50.033372 (+ 7727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:50.033390 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:50.033406 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:50.033453 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:50.036413 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:50.037072 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:50.037079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:50.037081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:50.037223 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:50.037681 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:50.037685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:50.037688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:50.038071 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:50.038338 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:50.038892 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 22:11:50.039068 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1188,"thread_start_us":1023,"threads_started":1}
W20260811 22:11:50.040037 14431 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50006
W20260811 22:11:50.040710 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 14234: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:11:50.040910 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 14234
I20260811 22:11:50.072525 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13616
I20260811 22:11:50.073751 13729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:50.192777 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 13616
2026-08-11T22:11:50Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9717 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:11:50 dist-test-slave-68xk krb5kdc[14704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:11:50 dist-test-slave-68xk krb5kdc[14704](info): set up 2 sockets
Aug 11 22:11:50 dist-test-slave-68xk krb5kdc[14704](info): commencing operation
krb5kdc: starting...
W20260811 22:11:51.515558 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.007s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:11:51 dist-test-slave-68xk krb5kdc[14704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:11:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:11:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:11:51.743103 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:40665
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:40665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:52.080724 14721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:52.081051 14721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:52.081211 14721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:52.088459 14721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:11:52.088606 14721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:52.088718 14721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:52.088814 14721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:11:52.088896 14721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:11:52.098670 14721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:40665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:40665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:52.100015 14721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:52.101387 14721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:52.113736 14729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:52.114022 14721 server_base.cc:1034] running on GCE node
W20260811 22:11:52.113708 14735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:52.113744 14730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:52.115408 14721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:52.116847 14721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:52.118085 14721 hybrid_clock.cc:648] HybridClock initialized: now 1786486312118041 us; error 38 us; skew 500 ppm
I20260811 22:11:52.118507 14721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:52 dist-test-slave-68xk krb5kdc[14704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486312, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:52.121681 14721 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:11:52.123373 14721 webserver.cc:460] Webserver started at http://127.10.173.190:41885/ using document root <none> and password file <none>
I20260811 22:11:52.124065 14721 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:52.124210 14721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:52.124532 14721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:52.127600 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f"
format_stamp: "Formatted at 2026-08-11 22:11:52 on dist-test-slave-68xk"
server_key: "af76632f2ef0eb8d9f59a0c716d6f681"
server_key_iv: "b13f80480079f2958f2a4bedc8573338"
server_key_version: "encryptionkey@0"
I20260811 22:11:52.128368 14721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f"
format_stamp: "Formatted at 2026-08-11 22:11:52 on dist-test-slave-68xk"
server_key: "af76632f2ef0eb8d9f59a0c716d6f681"
server_key_iv: "b13f80480079f2958f2a4bedc8573338"
server_key_version: "encryptionkey@0"
I20260811 22:11:52.133432 14721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:11:52.136968 14745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:52.138332 14721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:11:52.138541 14721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f"
format_stamp: "Formatted at 2026-08-11 22:11:52 on dist-test-slave-68xk"
server_key: "af76632f2ef0eb8d9f59a0c716d6f681"
server_key_iv: "b13f80480079f2958f2a4bedc8573338"
server_key_version: "encryptionkey@0"
I20260811 22:11:52.138749 14721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:52.170085 14721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:52.174542 14721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:52.174844 14721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:52.194716 14721 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:40665
I20260811 22:11:52.194747 14823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:40665 every 8 connection(s)
I20260811 22:11:52.196118 14721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:11:52.200091 14826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:52.204813 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 14721
I20260811 22:11:52.205078 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:11:52.205740 10934 external_mini_cluster.cc:1428] Setting key 855c490504dac1a7b5738aed3cfcdcab
I20260811 22:11:52.211417 14826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f: Bootstrap starting.
I20260811 22:11:52.215283 14826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:52.216601 14826 log.cc:824] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:52.220448 14826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f: No bootstrap required, opened a new log
Aug 11 22:11:52 dist-test-slave-68xk krb5kdc[14704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:52.228657 14826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40665 } }
I20260811 22:11:52.229065 14826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:52.229195 14826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81b5acc4172c40a6a00efc1b4f8b0e9f, State: Initialized, Role: FOLLOWER
I20260811 22:11:52.229964 14826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40665 } }
I20260811 22:11:52.230262 14826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:11:52.230427 14826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:11:52.230614 14826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:52.231701 14831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:52.207880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43938 (local address 127.10.173.190:40665)
0811 22:11:52.208779 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:52.208794 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:52.208839 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:11:52.210646 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:52.210669 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:52.211055 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:52.211377 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:52.213403 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:52.214781 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:52.216669 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:52.217922 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:52.218175 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:52.223361 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:52.223370 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:52.223386 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:52.223426 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:52.226864 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:52.227664 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:52.227672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:52.227679 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:52.227851 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:52.228258 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:52.228264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:52.228267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:52.228876 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:52.229186 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:52.229723 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 22:11:52.230188 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":400,"threads_started":1}
I20260811 22:11:52.233376 14826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40665 } }
I20260811 22:11:52.233871 14826 leader_election.cc:302] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81b5acc4172c40a6a00efc1b4f8b0e9f; no voters:
I20260811 22:11:52.234582 14826 leader_election.cc:288] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:11:52.234710 14833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:52.235405 14833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [term 1 LEADER]: Becoming Leader. State: Replica: 81b5acc4172c40a6a00efc1b4f8b0e9f, State: Running, Role: LEADER
I20260811 22:11:52.236011 14833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40665 } }
I20260811 22:11:52.236994 14826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:11:52.239252 14836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81b5acc4172c40a6a00efc1b4f8b0e9f. Latest consensus state: current_term: 1 leader_uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40665 } } }
I20260811 22:11:52.239809 14836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:52.239221 14838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81b5acc4172c40a6a00efc1b4f8b0e9f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40665 } } }
I20260811 22:11:52.240494 14838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:52.243723 14848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:11:52.249174 14848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:11:52.258826 14848 catalog_manager.cc:1347] Generated new cluster ID: 24fd4b0d00e6451da0f52edfbd893561
I20260811 22:11:52.258960 14848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:11:52.313829 14848 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:11:52.315261 14848 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:11:52.326341 14848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81b5acc4172c40a6a00efc1b4f8b0e9f: Generated new TSK 0
I20260811 22:11:52.327258 14848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:11:52.391053 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:40665
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:52.731320 14866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:52.731585 14866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:52.731745 14866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:52.739068 14866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:52.739176 14866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:52.739365 14866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:11:52.749575 14866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:40665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:52.750900 14866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:52.752254 14866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:52.765909 14874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:52.766000 14879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:52.765957 14875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:52.766970 14866 server_base.cc:1034] running on GCE node
I20260811 22:11:52.767580 14866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:52.768522 14866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:52.769757 14866 hybrid_clock.cc:648] HybridClock initialized: now 1786486312769689 us; error 60 us; skew 500 ppm
I20260811 22:11:52.770079 14866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:52 dist-test-slave-68xk krb5kdc[14704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486312, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:52.779604 14866 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:11:52.781167 14866 webserver.cc:460] Webserver started at http://127.10.173.129:43521/ using document root <none> and password file <none>
I20260811 22:11:52.781905 14866 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:52.782050 14866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:52.782370 14866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:52.785425 14866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "af3a507933c6423f94f219365b019332"
format_stamp: "Formatted at 2026-08-11 22:11:52 on dist-test-slave-68xk"
server_key: "287987ec4989208a96d876ad10782a72"
server_key_iv: "5dd6352f419e130b6d7415ce64a647ae"
server_key_version: "encryptionkey@0"
I20260811 22:11:52.786257 14866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "af3a507933c6423f94f219365b019332"
format_stamp: "Formatted at 2026-08-11 22:11:52 on dist-test-slave-68xk"
server_key: "287987ec4989208a96d876ad10782a72"
server_key_iv: "5dd6352f419e130b6d7415ce64a647ae"
server_key_version: "encryptionkey@0"
I20260811 22:11:52.791563 14866 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:11:52.794992 14891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:52.796224 14866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:11:52.796398 14866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "af3a507933c6423f94f219365b019332"
format_stamp: "Formatted at 2026-08-11 22:11:52 on dist-test-slave-68xk"
server_key: "287987ec4989208a96d876ad10782a72"
server_key_iv: "5dd6352f419e130b6d7415ce64a647ae"
server_key_version: "encryptionkey@0"
I20260811 22:11:52.796607 14866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:52.819197 14866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:52.823299 14866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:52.823602 14866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:52.824596 14866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:52.826033 14866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:52.826136 14866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:52.826283 14866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:52.826360 14866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:52.862584 14866 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:37403
I20260811 22:11:52.862675 15079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:37403 every 8 connection(s)
I20260811 22:11:52.863953 14866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:11:52.865271 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 14866
I20260811 22:11:52.865533 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:11:52.866533 10934 external_mini_cluster.cc:1428] Setting key 0253adc663a30aa0bcf25c873a520058
Aug 11 22:11:52 dist-test-slave-68xk krb5kdc[14704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486312, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:52.890359 15084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:52.867247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:48821 (local address 127.10.173.190:40665)
0811 22:11:52.867854 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:52.867859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:52.869004 (+ 1145us) server_negotiation.cc:407] Connection header received
0811 22:11:52.872204 (+ 3200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:52.872214 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:52.872277 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:52.872373 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:52.874722 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:52.875465 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:52.877936 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:52.878644 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:52.878817 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:52.882976 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:52.882986 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:52.882994 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:52.883041 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:52.886809 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:52.887640 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:52.887649 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:52.887654 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:52.887839 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:52.888288 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:52.888296 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:52.888301 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:52.888697 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:52.888878 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:52.889634 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 22:11:52.889910 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":299,"threads_started":1}
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
I20260811 22:11:52.891134 15085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:52.867855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:40665 (local address 127.10.173.129:48821)
0811 22:11:52.868821 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:52.868874 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:52.871900 (+ 3026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:52.872671 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:52.872687 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:52.873593 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:52.874523 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:52.874553 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:52.875642 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:52.875651 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:52.877774 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:52.877787 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:52.878901 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:52.878907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:52.879123 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:52.879921 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:52.879947 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:52.882730 (+ 2783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:52.886993 (+ 4263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:52.887001 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:52.887015 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:52.887481 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:52.887964 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:52.887967 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:52.887969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:52.888151 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:52.888858 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:52.888865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:52.889220 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 22:11:52.889455 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 22:11:52.889883 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":345,"threads_started":1}
I20260811 22:11:52.892748 15080 heartbeater.cc:347] Connected to a master server at 127.10.173.190:40665
I20260811 22:11:52.893067 15080 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:52.893790 15080 heartbeater.cc:511] Master 127.10.173.190:40665 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
I20260811 22:11:52.896039 14773 ts_manager.cc:194] Registered new tserver with Master: af3a507933c6423f94f219365b019332 (127.10.173.129:37403)
I20260811 22:11:52.897500 14773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:48821
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:11:52.922911 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:40665
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:53.253252 15090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:53.253587 15090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:53.253685 15090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:53.261121 15090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:53.261241 15090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:53.261358 15090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:11:53.271140 15090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:40665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:53.272440 15090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:53.273761 15090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:53.287833 15105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:53.287904 15101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:53.287869 15097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:53.288717 15090 server_base.cc:1034] running on GCE node
I20260811 22:11:53.289429 15090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:53.290514 15090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:53.291749 15090 hybrid_clock.cc:648] HybridClock initialized: now 1786486313291662 us; error 67 us; skew 500 ppm
I20260811 22:11:53.292085 15090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:53 dist-test-slave-68xk krb5kdc[14704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486313, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:53.295255 15090 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:11:53.296700 15090 webserver.cc:460] Webserver started at http://127.10.173.130:38461/ using document root <none> and password file <none>
I20260811 22:11:53.297626 15090 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:53.297785 15090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:53.298053 15090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:53.301241 15090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "19cb07edb3274da29bfe69a02c942233"
format_stamp: "Formatted at 2026-08-11 22:11:53 on dist-test-slave-68xk"
server_key: "0c9975c12fd727b8296a6b27ebc0f3a4"
server_key_iv: "3814d20ae8ab3fa2efa9f57588aad771"
server_key_version: "encryptionkey@0"
I20260811 22:11:53.302021 15090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "19cb07edb3274da29bfe69a02c942233"
format_stamp: "Formatted at 2026-08-11 22:11:53 on dist-test-slave-68xk"
server_key: "0c9975c12fd727b8296a6b27ebc0f3a4"
server_key_iv: "3814d20ae8ab3fa2efa9f57588aad771"
server_key_version: "encryptionkey@0"
I20260811 22:11:53.307420 15090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 22:11:53.310904 15115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:53.312201 15090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:11:53.312384 15090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "19cb07edb3274da29bfe69a02c942233"
format_stamp: "Formatted at 2026-08-11 22:11:53 on dist-test-slave-68xk"
server_key: "0c9975c12fd727b8296a6b27ebc0f3a4"
server_key_iv: "3814d20ae8ab3fa2efa9f57588aad771"
server_key_version: "encryptionkey@0"
I20260811 22:11:53.312583 15090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:53.324980 15090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:53.329330 15090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:53.329623 15090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:53.330681 15090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:53.332093 15090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:53.332166 15090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:53.332324 15090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:53.332391 15090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:53.367014 15090 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:34011
I20260811 22:11:53.367234 15300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:34011 every 8 connection(s)
I20260811 22:11:53.368294 15090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:11:53.373332 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 15090
I20260811 22:11:53.373543 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:11:53.374078 10934 external_mini_cluster.cc:1428] Setting key 26b35feb05fd0d920340410dc1ead98e
Aug 11 22:11:53 dist-test-slave-68xk krb5kdc[14704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486313, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:53.391876 15084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:53.370925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:39117 (local address 127.10.173.190:40665)
0811 22:11:53.371107 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:53.371112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:53.372274 (+ 1162us) server_negotiation.cc:407] Connection header received
0811 22:11:53.374766 (+ 2492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:53.374773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:53.374875 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:53.375000 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:53.376850 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:53.377523 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:53.380027 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:53.380441 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:53.380533 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:53.384766 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:53.384776 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:53.384784 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:53.384830 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:53.388367 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:53.389124 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:53.389132 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:53.389139 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:53.389318 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:53.389759 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:53.389766 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:53.389772 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:53.390209 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:53.390374 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:53.391142 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 22:11:53.391440 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:11:53.392671 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:53.371389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:40665 (local address 127.10.173.130:39117)
0811 22:11:53.372108 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:53.372146 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:53.374436 (+ 2290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:53.375274 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:53.375289 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:53.375815 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:53.376661 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:53.376688 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:53.377686 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:53.377695 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:53.379890 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:53.379907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:53.380670 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:53.380680 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:53.381020 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:53.381790 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:53.381825 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:53.384525 (+ 2700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:53.388528 (+ 4003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:53.388536 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:53.388549 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:53.388981 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:53.389439 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:53.389443 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:53.389445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:53.389628 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:53.390343 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:53.390349 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:53.390767 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 22:11:53.390992 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 22:11:53.391393 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":283,"threads_started":1}
I20260811 22:11:53.394256 15306 heartbeater.cc:347] Connected to a master server at 127.10.173.190:40665
I20260811 22:11:53.394613 15306 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:53.395387 15306 heartbeater.cc:511] Master 127.10.173.190:40665 requested a full tablet report, sending...
I20260811 22:11:53.396905 14773 ts_manager.cc:194] Registered new tserver with Master: 19cb07edb3274da29bfe69a02c942233 (127.10.173.130:34011)
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
I20260811 22:11:53.397805 14773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:39117
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:11:53.430048 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:40665
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:35721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:53.760360 15313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:53.760699 15313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:53.760877 15313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:53.768007 15313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:53.768177 15313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:53.768352 15313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:11:53.778086 15313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:40665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:53.779464 15313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:53.780771 15313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:53.793926 15323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:53.793891 15326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:53.794000 15321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:53.794761 15313 server_base.cc:1034] running on GCE node
I20260811 22:11:53.795420 15313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:53.796379 15313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:53.797610 15313 hybrid_clock.cc:648] HybridClock initialized: now 1786486313797563 us; error 34 us; skew 500 ppm
I20260811 22:11:53.797973 15313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:53 dist-test-slave-68xk krb5kdc[14704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486313, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:53.801509 15313 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:11:53.803184 15313 webserver.cc:460] Webserver started at http://127.10.173.131:41373/ using document root <none> and password file <none>
I20260811 22:11:53.803894 15313 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:53.804030 15313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:53.804366 15313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:53.807415 15313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "9c29ce391a224a22bfcb9817bb843870"
format_stamp: "Formatted at 2026-08-11 22:11:53 on dist-test-slave-68xk"
server_key: "96de86ac7afee1160c05c659153c59f7"
server_key_iv: "b59d4f9c99649a017a748d2601f93a86"
server_key_version: "encryptionkey@0"
I20260811 22:11:53.808202 15313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "9c29ce391a224a22bfcb9817bb843870"
format_stamp: "Formatted at 2026-08-11 22:11:53 on dist-test-slave-68xk"
server_key: "96de86ac7afee1160c05c659153c59f7"
server_key_iv: "b59d4f9c99649a017a748d2601f93a86"
server_key_version: "encryptionkey@0"
I20260811 22:11:53.813280 15313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:11:53.816764 15341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:53.818157 15313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:11:53.818349 15313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "9c29ce391a224a22bfcb9817bb843870"
format_stamp: "Formatted at 2026-08-11 22:11:53 on dist-test-slave-68xk"
server_key: "96de86ac7afee1160c05c659153c59f7"
server_key_iv: "b59d4f9c99649a017a748d2601f93a86"
server_key_version: "encryptionkey@0"
I20260811 22:11:53.818656 15313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:53.851811 15313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:53.856112 15313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:53.856433 15313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:53.857633 15313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:53.859002 15313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:53.859107 15313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:53.859270 15313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:53.859354 15313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:53.894116 15313 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:33487
I20260811 22:11:53.894164 15520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:33487 every 8 connection(s)
I20260811 22:11:53.895439 15313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:11:53.900611 15080 heartbeater.cc:503] Master 127.10.173.190:40665 was elected leader, sending a full tablet report...
I20260811 22:11:53.901701 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 15313
I20260811 22:11:53.901901 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:11:53.902503 10934 external_mini_cluster.cc:1428] Setting key bcf4ac8650d4cb3c262fec733f1673dd
Aug 11 22:11:53 dist-test-slave-68xk krb5kdc[14704](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486313, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:53.917824 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:53.897990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:60863 (local address 127.10.173.190:40665)
0811 22:11:53.898512 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:53.898519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:53.899773 (+ 1254us) server_negotiation.cc:407] Connection header received
0811 22:11:53.902699 (+ 2926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:53.902704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:53.902793 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:53.902913 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:53.904964 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:53.905910 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:53.907729 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:53.908184 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:53.908327 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:53.911906 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:53.911911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:53.911915 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:53.911949 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:53.914518 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:53.915285 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:53.915295 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:53.915298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:53.915415 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:53.915881 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:53.915885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:53.915888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:53.916292 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:53.916444 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:53.917192 (+ 748us) server_negotiation.cc:299] Negotiation successful
0811 22:11:53.917433 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":236,"threads_started":1}
I20260811 22:11:53.918962 15528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:53.898670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:40665 (local address 127.10.173.131:60863)
0811 22:11:53.899437 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:53.899510 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:53.902193 (+ 2683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:53.903229 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:53.903246 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:53.904005 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:53.904782 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:53.904803 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:53.906094 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:53.906106 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:53.907568 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:53.907582 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:53.908444 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:53.908456 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:53.908721 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:53.909407 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:53.909434 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:53.911672 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:53.914660 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:53.914670 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:53.914688 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:53.915138 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:53.915529 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:53.915534 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:53.915538 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:53.915750 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:53.916423 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:53.916431 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:53.916798 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 22:11:53.917071 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 22:11:53.917766 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":327,"threads_started":1}
I20260811 22:11:53.920413 15521 heartbeater.cc:347] Connected to a master server at 127.10.173.190:40665
I20260811 22:11:53.920683 15521 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:53.921377 15521 heartbeater.cc:511] Master 127.10.173.190:40665 requested a full tablet report, sending...
I20260811 22:11:53.923041 14773 ts_manager.cc:194] Registered new tserver with Master: 9c29ce391a224a22bfcb9817bb843870 (127.10.173.131:33487)
I20260811 22:11:53.923966 14773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:60863
I20260811 22:11:53.930261 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:11:53.967567 15527 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:11:53.964932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.10.173.190:40665)
0811 22:11:53.965217 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:53.965228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:53.965586 (+ 358us) server_negotiation.cc:407] Connection header received
0811 22:11:53.965731 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:53.965735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:53.965820 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:53.965913 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:53.967279 (+ 1366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43948: BlockingRecv error: recv got EOF from 127.0.0.1:43948 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
I20260811 22:11:53.971174 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 14866
W20260811 22:11:53.974028 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 14866: Not authorized: Client connection negotiation failed: client connection to 127.10.173.129:37403: server requires authentication, but client does not have Kerberos credentials available
I20260811 22:11:53.974232 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 14866
I20260811 22:11:53.996381 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 15090
W20260811 22:11:53.999567 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 15090: Not authorized: Client connection negotiation failed: client connection to 127.10.173.130:34011: server requires authentication, but client does not have Kerberos credentials available
I20260811 22:11:53.999776 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 15090
I20260811 22:11:54.028724 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 15313
W20260811 22:11:54.031889 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 15313: Not authorized: Client connection negotiation failed: client connection to 127.10.173.131:33487: server requires authentication, but client does not have Kerberos credentials available
I20260811 22:11:54.032042 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 15313
I20260811 22:11:54.061686 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 14721
I20260811 22:11:54.063040 14816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:54.167922 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 14721
2026-08-11T22:11:54Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3971 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:11:54 dist-test-slave-68xk krb5kdc[15549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:11:54 dist-test-slave-68xk krb5kdc[15549](info): set up 2 sockets
Aug 11 22:11:54 dist-test-slave-68xk krb5kdc[15549](info): commencing operation
krb5kdc: starting...
W20260811 22:11:55.496466 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:11:55 dist-test-slave-68xk krb5kdc[15549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:11:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:11:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:11:55.722645 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:40529
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:42595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:40529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:56.061698 15571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:56.062107 15571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:56.062284 15571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:56.069585 15571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:11:56.069733 15571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:56.069875 15571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:56.069999 15571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:11:56.070086 15571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:11:56.080354 15571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:42595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:40529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:40529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:56.081621 15571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:56.083074 15571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:56.095467 15581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:56.095345 15584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:56.095342 15582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:56.096273 15571 server_base.cc:1034] running on GCE node
I20260811 22:11:56.097061 15571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:56.098551 15571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:56.099797 15571 hybrid_clock.cc:648] HybridClock initialized: now 1786486316099753 us; error 54 us; skew 500 ppm
I20260811 22:11:56.100179 15571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:56 dist-test-slave-68xk krb5kdc[15549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486316, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:56.103508 15571 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:11:56.104967 15571 webserver.cc:460] Webserver started at http://127.10.173.190:45975/ using document root <none> and password file <none>
I20260811 22:11:56.105691 15571 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:56.105847 15571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:56.106189 15571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:56.109297 15571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "b8f95eeff0cc40c685634f925715c343"
format_stamp: "Formatted at 2026-08-11 22:11:56 on dist-test-slave-68xk"
server_key: "d886f71d3f1c105bd260e31d26acbecc"
server_key_iv: "5e3bbfad818e5a2bb611e8c05f4e6d77"
server_key_version: "encryptionkey@0"
I20260811 22:11:56.110132 15571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "b8f95eeff0cc40c685634f925715c343"
format_stamp: "Formatted at 2026-08-11 22:11:56 on dist-test-slave-68xk"
server_key: "d886f71d3f1c105bd260e31d26acbecc"
server_key_iv: "5e3bbfad818e5a2bb611e8c05f4e6d77"
server_key_version: "encryptionkey@0"
I20260811 22:11:56.115482 15571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.002s
I20260811 22:11:56.119122 15596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:56.120570 15571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:11:56.120777 15571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "b8f95eeff0cc40c685634f925715c343"
format_stamp: "Formatted at 2026-08-11 22:11:56 on dist-test-slave-68xk"
server_key: "d886f71d3f1c105bd260e31d26acbecc"
server_key_iv: "5e3bbfad818e5a2bb611e8c05f4e6d77"
server_key_version: "encryptionkey@0"
I20260811 22:11:56.121001 15571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:56.149987 15571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:56.155117 15571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:56.155450 15571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:56.175983 15571 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:40529
I20260811 22:11:56.176007 15678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:40529 every 8 connection(s)
I20260811 22:11:56.177527 15571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:11:56.181392 15679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:56.183826 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 15571
I20260811 22:11:56.184075 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:11:56.184886 10934 external_mini_cluster.cc:1428] Setting key f2acdd3715363a71f84ac9370c8694e6
I20260811 22:11:56.194505 15679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343: Bootstrap starting.
I20260811 22:11:56.198218 15679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:56.199281 15679 log.cc:824] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:11:56 dist-test-slave-68xk krb5kdc[15549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:56.202227 15679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343: No bootstrap required, opened a new log
I20260811 22:11:56.209506 15683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:56.187030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.10.173.190:40529)
0811 22:11:56.187893 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:56.187906 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:56.187949 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:11:56.189616 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:56.189639 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:56.190009 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:56.190340 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:56.192219 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:56.193573 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:56.195419 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:56.196113 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:56.196324 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:56.201032 (+ 4708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:56.201071 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:56.201087 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:56.201127 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:56.204864 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:56.205821 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:56.205828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:56.205835 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:56.206020 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:56.206523 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:56.206528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:56.206530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:56.207166 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:56.207419 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:56.207931 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 22:11:56.208437 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":414,"threads_started":1}
I20260811 22:11:56.211042 15679 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f95eeff0cc40c685634f925715c343" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40529 } }
I20260811 22:11:56.211398 15679 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:56.211509 15679 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f95eeff0cc40c685634f925715c343, State: Initialized, Role: FOLLOWER
I20260811 22:11:56.212090 15679 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f95eeff0cc40c685634f925715c343" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40529 } }
I20260811 22:11:56.212339 15679 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:11:56.212481 15679 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:11:56.212662 15679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:56.213838 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:11:56.215164 15679 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f95eeff0cc40c685634f925715c343" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40529 } }
I20260811 22:11:56.215634 15679 leader_election.cc:302] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8f95eeff0cc40c685634f925715c343; no voters:
I20260811 22:11:56.216271 15679 leader_election.cc:288] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:11:56.216428 15686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:56.218359 15679 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:11:56.218710 15686 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [term 1 LEADER]: Becoming Leader. State: Replica: b8f95eeff0cc40c685634f925715c343, State: Running, Role: LEADER
I20260811 22:11:56.219427 15686 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f95eeff0cc40c685634f925715c343" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40529 } }
I20260811 22:11:56.222767 15687 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8f95eeff0cc40c685634f925715c343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f95eeff0cc40c685634f925715c343" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40529 } } }
I20260811 22:11:56.222800 15688 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8f95eeff0cc40c685634f925715c343. Latest consensus state: current_term: 1 leader_uuid: "b8f95eeff0cc40c685634f925715c343" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f95eeff0cc40c685634f925715c343" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40529 } } }
I20260811 22:11:56.223305 15688 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:56.223310 15687 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343 [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:56.224745 15696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:11:56.234364 15696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:11:56.248255 15696 catalog_manager.cc:1347] Generated new cluster ID: 7661ee3814d743058de3df65ccf98754
I20260811 22:11:56.248365 15696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:11:56.292711 15683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:56.281385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48026 (local address 127.10.173.190:40529)
0811 22:11:56.281562 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:56.281568 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:56.283068 (+ 1500us) server_negotiation.cc:407] Connection header received
0811 22:11:56.283260 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:56.283267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:56.283361 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:56.283498 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:56.284734 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:56.286053 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:56.287860 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:56.288623 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:56.288788 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:56.290590 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:56.290599 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:56.290604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:11:56.291726 (+ 1122us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:56.291838 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:11:56.291874 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:56.291971 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:56.292265 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 22:11:56.292365 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:11:56.301609 15696 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:11:56.303746 15696 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:11:56.313396 15696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8f95eeff0cc40c685634f925715c343: Generated new TSK 0
I20260811 22:11:56.314545 15696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:11:56.329066 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 15571
I20260811 22:11:56.330399 15668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:11:56.426149 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 15571
2026-08-11T22:11:56Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2252 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:11:56 dist-test-slave-68xk krb5kdc[15731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:11:56 dist-test-slave-68xk krb5kdc[15731](info): set up 2 sockets
Aug 11 22:11:56 dist-test-slave-68xk krb5kdc[15731](info): commencing operation
krb5kdc: starting...
W20260811 22:11:57.745193 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.007s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:11:57 dist-test-slave-68xk krb5kdc[15731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:11:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:11:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:11:57.966832 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:36311
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:41041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:36311
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:58.298921 15749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:58.299227 15749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:58.299320 15749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:58.306543 15749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:11:58.306654 15749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:58.306703 15749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:58.306761 15749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:11:58.306800 15749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:11:58.316557 15749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:41041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:36311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:36311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:58.317984 15749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:58.319324 15749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:58.331624 15756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:58.331593 15760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:58.331692 15757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:58.336179 15749 server_base.cc:1034] running on GCE node
I20260811 22:11:58.336946 15749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:58.338352 15749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:58.339632 15749 hybrid_clock.cc:648] HybridClock initialized: now 1786486318339582 us; error 45 us; skew 500 ppm
I20260811 22:11:58.339985 15749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:58 dist-test-slave-68xk krb5kdc[15731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486318, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:58.346517 15749 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:11:58.348143 15749 webserver.cc:460] Webserver started at http://127.10.173.190:33927/ using document root <none> and password file <none>
I20260811 22:11:58.348865 15749 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:58.349013 15749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:58.349349 15749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:58.352411 15749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "bab59f5453754f00805956cc011e20bc"
format_stamp: "Formatted at 2026-08-11 22:11:58 on dist-test-slave-68xk"
server_key: "9927210a4759dcef8c6999ad9fec516a"
server_key_iv: "78fe141f725dd408f9b568d335c68129"
server_key_version: "encryptionkey@0"
I20260811 22:11:58.353186 15749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "bab59f5453754f00805956cc011e20bc"
format_stamp: "Formatted at 2026-08-11 22:11:58 on dist-test-slave-68xk"
server_key: "9927210a4759dcef8c6999ad9fec516a"
server_key_iv: "78fe141f725dd408f9b568d335c68129"
server_key_version: "encryptionkey@0"
I20260811 22:11:58.358574 15749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 22:11:58.361986 15773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:58.363399 15749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:11:58.363590 15749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "bab59f5453754f00805956cc011e20bc"
format_stamp: "Formatted at 2026-08-11 22:11:58 on dist-test-slave-68xk"
server_key: "9927210a4759dcef8c6999ad9fec516a"
server_key_iv: "78fe141f725dd408f9b568d335c68129"
server_key_version: "encryptionkey@0"
I20260811 22:11:58.363811 15749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:58.379761 15749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:58.383924 15749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:58.384217 15749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:58.403292 15749 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:36311
I20260811 22:11:58.403343 15850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:36311 every 8 connection(s)
I20260811 22:11:58.404668 15749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:11:58.408562 15853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:58.409225 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 15749
I20260811 22:11:58.409443 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:11:58.410326 10934 external_mini_cluster.cc:1428] Setting key b30d0b206d73f6c5a643b387b5c67b40
I20260811 22:11:58.423456 15853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc: Bootstrap starting.
Aug 11 22:11:58 dist-test-slave-68xk krb5kdc[15731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:58.427649 15853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:58.428948 15853 log.cc:824] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:58.432497 15853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc: No bootstrap required, opened a new log
I20260811 22:11:58.435973 15859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:58.412628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.10.173.190:36311)
0811 22:11:58.413529 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:58.413544 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:58.413594 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:11:58.415484 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:58.415509 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:58.415941 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:58.416301 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:58.418106 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:58.419479 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:58.421298 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:58.421939 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:58.422064 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:58.427870 (+ 5806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:58.427882 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:58.427903 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:58.427954 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:58.431517 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:58.432432 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:58.432438 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:58.432444 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:58.432690 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:58.433219 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:58.433222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:58.433224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:58.433773 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:58.434007 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:58.434551 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 22:11:58.434920 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":402,"threads_started":1}
I20260811 22:11:58.440834 15853 raft_consensus.cc:348] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab59f5453754f00805956cc011e20bc" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36311 } }
I20260811 22:11:58.441203 15853 raft_consensus.cc:374] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:58.441310 15853 raft_consensus.cc:729] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bab59f5453754f00805956cc011e20bc, State: Initialized, Role: FOLLOWER
I20260811 22:11:58.441998 15853 consensus_queue.cc:260] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab59f5453754f00805956cc011e20bc" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36311 } }
I20260811 22:11:58.442281 15853 raft_consensus.cc:388] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:11:58.442453 15853 raft_consensus.cc:482] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:11:58.442649 15853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:58.444909 15853 raft_consensus.cc:504] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab59f5453754f00805956cc011e20bc" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36311 } }
I20260811 22:11:58.445461 15853 leader_election.cc:302] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bab59f5453754f00805956cc011e20bc; no voters:
I20260811 22:11:58.446125 15853 leader_election.cc:288] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:11:58.446266 15862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:58.447032 15862 raft_consensus.cc:686] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [term 1 LEADER]: Becoming Leader. State: Replica: bab59f5453754f00805956cc011e20bc, State: Running, Role: LEADER
I20260811 22:11:58.447599 15862 consensus_queue.cc:237] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab59f5453754f00805956cc011e20bc" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36311 } }
I20260811 22:11:58.448271 15853 sys_catalog.cc:564] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:11:58.450484 15863 sys_catalog.cc:455] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bab59f5453754f00805956cc011e20bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab59f5453754f00805956cc011e20bc" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36311 } } }
I20260811 22:11:58.450811 15864 sys_catalog.cc:455] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader bab59f5453754f00805956cc011e20bc. Latest consensus state: current_term: 1 leader_uuid: "bab59f5453754f00805956cc011e20bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bab59f5453754f00805956cc011e20bc" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36311 } } }
I20260811 22:11:58.451128 15863 sys_catalog.cc:458] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:58.451833 15864 sys_catalog.cc:458] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:58.453313 15875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:11:58.458848 15875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:11:58.468122 15875 catalog_manager.cc:1347] Generated new cluster ID: 703544aed9894597975aecb55b3f05a0
I20260811 22:11:58.468297 15875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:11:58.486496 15875 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:11:58.487787 15875 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:11:58.495714 15875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bab59f5453754f00805956cc011e20bc: Generated new TSK 0
I20260811 22:11:58.496783 15875 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:11:58.561790 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36311
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:41041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:58.909966 15893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:58.910336 15893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:58.910537 15893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:58.917938 15893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:58.918118 15893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:58.918304 15893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:11:58.928351 15893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:41041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:58.929788 15893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:58.931247 15893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:58.945207 15903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:58.945293 15908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:58.945218 15904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:58.946507 15893 server_base.cc:1034] running on GCE node
I20260811 22:11:58.947149 15893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:58.948144 15893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:58.949404 15893 hybrid_clock.cc:648] HybridClock initialized: now 1786486318949342 us; error 45 us; skew 500 ppm
I20260811 22:11:58.949791 15893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:58 dist-test-slave-68xk krb5kdc[15731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486318, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:58.954033 15893 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:11:58.955842 15893 webserver.cc:460] Webserver started at http://127.10.173.129:38459/ using document root <none> and password file <none>
I20260811 22:11:58.956619 15893 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:58.956789 15893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:58.957139 15893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:58.960609 15893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "1681107e7534489f8c7b73161ede2459"
format_stamp: "Formatted at 2026-08-11 22:11:58 on dist-test-slave-68xk"
server_key: "997db14e58e19f590e13ae6826287d5b"
server_key_iv: "d21c4b191fdaa8ca5a19c4c9bd049bc4"
server_key_version: "encryptionkey@0"
I20260811 22:11:58.961511 15893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "1681107e7534489f8c7b73161ede2459"
format_stamp: "Formatted at 2026-08-11 22:11:58 on dist-test-slave-68xk"
server_key: "997db14e58e19f590e13ae6826287d5b"
server_key_iv: "d21c4b191fdaa8ca5a19c4c9bd049bc4"
server_key_version: "encryptionkey@0"
I20260811 22:11:58.967295 15893 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 22:11:58.971238 15917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:58.972688 15893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:11:58.972890 15893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "1681107e7534489f8c7b73161ede2459"
format_stamp: "Formatted at 2026-08-11 22:11:58 on dist-test-slave-68xk"
server_key: "997db14e58e19f590e13ae6826287d5b"
server_key_iv: "d21c4b191fdaa8ca5a19c4c9bd049bc4"
server_key_version: "encryptionkey@0"
I20260811 22:11:58.973114 15893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:58.996074 15893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:59.001039 15893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:59.001418 15893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:59.002593 15893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:59.004078 15893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:59.004199 15893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:59.004379 15893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:59.004460 15893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:59.044533 15893 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:44087
I20260811 22:11:59.044574 16092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:44087 every 8 connection(s)
I20260811 22:11:59.045969 15893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:11:59.055987 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 15893
I20260811 22:11:59.056252 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:11:59.057171 10934 external_mini_cluster.cc:1428] Setting key b3579b6472cbb573243984420c025771
Aug 11 22:11:59 dist-test-slave-68xk krb5kdc[15731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486318, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:59.069350 16097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:59.048806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:48177 (local address 127.10.173.190:36311)
0811 22:11:59.049363 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:59.049370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:59.050454 (+ 1084us) server_negotiation.cc:407] Connection header received
0811 22:11:59.052270 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:59.052274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:59.052361 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:59.052518 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:59.054157 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:59.055049 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:59.056742 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:59.057476 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:59.057649 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:59.061771 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:59.061781 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:59.061790 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:59.061841 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:59.065823 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:59.066614 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:59.066623 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:59.066630 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:59.066801 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:59.067362 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:59.067370 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:59.067377 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:59.067794 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:59.067962 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:59.068857 (+ 895us) server_negotiation.cc:299] Negotiation successful
0811 22:11:59.069036 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":238,"threads_started":1}
I20260811 22:11:59.071008 16098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:59.049479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36311 (local address 127.10.173.129:48177)
0811 22:11:59.050256 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:59.050301 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:59.051959 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:59.052735 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:59.052744 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:59.053310 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:59.053983 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:59.054005 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:59.055185 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:59.055188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:59.056573 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:59.056585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:59.057701 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:59.057705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:59.057938 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:59.058758 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:59.058793 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:59.061540 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:59.065982 (+ 4442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:59.065989 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:59.066002 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:59.066495 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:59.066925 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:59.066928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:59.066930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:59.067202 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:59.067936 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:59.067942 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:59.068441 (+ 499us) client_negotiation.cc:769] Sending connection context
0811 22:11:59.068775 (+ 334us) client_negotiation.cc:240] Negotiation successful
0811 22:11:59.069371 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":333,"threads_started":1}
I20260811 22:11:59.072664 16094 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36311
I20260811 22:11:59.073009 16094 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:59.073838 16094 heartbeater.cc:511] Master 127.10.173.190:36311 requested a full tablet report, sending...
I20260811 22:11:59.076115 15799 ts_manager.cc:194] Registered new tserver with Master: 1681107e7534489f8c7b73161ede2459 (127.10.173.129:44087)
I20260811 22:11:59.078019 15799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:48177
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:11:59.111508 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36311
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:41041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:59.444897 16101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:59.445230 16101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:59.445415 16101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:59.453002 16101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:59.453179 16101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:59.453368 16101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:11:59.463390 16101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:41041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:11:59.464725 16101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:59.466105 16101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:59.479831 16107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:59.479864 16108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:59.480095 16113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:59.480731 16101 server_base.cc:1034] running on GCE node
I20260811 22:11:59.481403 16101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:59.482386 16101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:59.483680 16101 hybrid_clock.cc:648] HybridClock initialized: now 1786486319483626 us; error 43 us; skew 500 ppm
I20260811 22:11:59.484063 16101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:11:59 dist-test-slave-68xk krb5kdc[15731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486319, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:59.487524 16101 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:11:59.489054 16101 webserver.cc:460] Webserver started at http://127.10.173.130:39977/ using document root <none> and password file <none>
I20260811 22:11:59.489719 16101 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:59.489825 16101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:59.490093 16101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:11:59.493158 16101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "a7ab9a7e128348d4a83d88cc9f7426fa"
format_stamp: "Formatted at 2026-08-11 22:11:59 on dist-test-slave-68xk"
server_key: "acc8624a5ff3383fe748cdf221f68479"
server_key_iv: "ed9d1cddde7f7f16d12b61d752c05e6e"
server_key_version: "encryptionkey@0"
I20260811 22:11:59.493921 16101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "a7ab9a7e128348d4a83d88cc9f7426fa"
format_stamp: "Formatted at 2026-08-11 22:11:59 on dist-test-slave-68xk"
server_key: "acc8624a5ff3383fe748cdf221f68479"
server_key_iv: "ed9d1cddde7f7f16d12b61d752c05e6e"
server_key_version: "encryptionkey@0"
I20260811 22:11:59.499307 16101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 22:11:59.502912 16123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:59.504254 16101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:11:59.504460 16101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "a7ab9a7e128348d4a83d88cc9f7426fa"
format_stamp: "Formatted at 2026-08-11 22:11:59 on dist-test-slave-68xk"
server_key: "acc8624a5ff3383fe748cdf221f68479"
server_key_iv: "ed9d1cddde7f7f16d12b61d752c05e6e"
server_key_version: "encryptionkey@0"
I20260811 22:11:59.504669 16101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:59.523103 16101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:59.527434 16101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:59.527730 16101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:59.528992 16101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:59.530859 16101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:59.530961 16101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:59.531093 16101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:59.531179 16101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:59.583598 16101 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:43677
I20260811 22:11:59.583730 16283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:43677 every 8 connection(s)
I20260811 22:11:59.585458 16101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:11:59.595532 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 16101
I20260811 22:11:59.595728 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:11:59.596359 10934 external_mini_cluster.cc:1428] Setting key 86e2486075d91215cd62e7d80bdcae53
Aug 11 22:11:59 dist-test-slave-68xk krb5kdc[15731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486319, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:11:59.609503 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:59.588648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:36601 (local address 127.10.173.190:36311)
0811 22:11:59.589106 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:59.589111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:59.590371 (+ 1260us) server_negotiation.cc:407] Connection header received
0811 22:11:59.593403 (+ 3032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:59.593410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:59.593540 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:59.593658 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:59.596052 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:59.597138 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:59.599083 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:59.599512 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:59.599651 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:59.603491 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:59.603497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:59.603500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:59.603532 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:59.606170 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:59.606992 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:59.607000 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:59.607008 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:59.607129 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:59.607566 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:59.607571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:59.607576 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:59.607846 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:59.608003 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:59.608872 (+ 869us) server_negotiation.cc:299] Negotiation successful
0811 22:11:59.609105 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":242,"threads_started":1}
I20260811 22:11:59.610612 16287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:59.589257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36311 (local address 127.10.173.130:36601)
0811 22:11:59.590155 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:59.590227 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:59.593071 (+ 2844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:59.593964 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:59.593977 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:59.594885 (+ 908us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:59.595832 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:59.595865 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:59.597342 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:59.597347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:59.598941 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:59.598955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:59.599749 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:59.599757 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:59.599994 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:59.600699 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:59.600727 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:59.603230 (+ 2503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:59.606331 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:59.606340 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:59.606359 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:59.606860 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:59.607243 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:59.607247 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:59.607252 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:59.607442 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:59.607976 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:59.607984 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:59.608409 (+ 425us) client_negotiation.cc:769] Sending connection context
0811 22:11:59.608761 (+ 352us) client_negotiation.cc:240] Negotiation successful
0811 22:11:59.609327 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":315,"threads_started":1}
I20260811 22:11:59.612113 16284 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36311
I20260811 22:11:59.612418 16284 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:59.613147 16284 heartbeater.cc:511] Master 127.10.173.190:36311 requested a full tablet report, sending...
I20260811 22:11:59.614692 15799 ts_manager.cc:194] Registered new tserver with Master: a7ab9a7e128348d4a83d88cc9f7426fa (127.10.173.130:43677)
I20260811 22:11:59.615991 15799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:36601
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:11:59.651239 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36311
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:41041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:00.035100 16290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:00.035440 16290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:00.035615 16290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:00.043008 16290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:00.043175 16290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:00.043354 16290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:12:00.060530 16290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:41041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:00.062201 16290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:00.064111 16290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:00.080819 16299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:00.081092 16298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:00.081187 16094 heartbeater.cc:503] Master 127.10.173.190:36311 was elected leader, sending a full tablet report...
W20260811 22:12:00.082244 16301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:00.083673 16290 server_base.cc:1034] running on GCE node
I20260811 22:12:00.084350 16290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:00.085435 16290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:00.086671 16290 hybrid_clock.cc:648] HybridClock initialized: now 1786486320086610 us; error 54 us; skew 500 ppm
I20260811 22:12:00.087025 16290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:00 dist-test-slave-68xk krb5kdc[15731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486320, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:00.090732 16290 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:12:00.092875 16290 webserver.cc:460] Webserver started at http://127.10.173.131:42709/ using document root <none> and password file <none>
I20260811 22:12:00.093966 16290 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:00.094130 16290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:00.094617 16290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:00.099843 16290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "beda17177b6445208b7c63b4f5b56356"
format_stamp: "Formatted at 2026-08-11 22:12:00 on dist-test-slave-68xk"
server_key: "3738044cbd96580eff4937dfd0a57590"
server_key_iv: "d82f744ccb26d6055d75abb4cef3e922"
server_key_version: "encryptionkey@0"
I20260811 22:12:00.101085 16290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "beda17177b6445208b7c63b4f5b56356"
format_stamp: "Formatted at 2026-08-11 22:12:00 on dist-test-slave-68xk"
server_key: "3738044cbd96580eff4937dfd0a57590"
server_key_iv: "d82f744ccb26d6055d75abb4cef3e922"
server_key_version: "encryptionkey@0"
I20260811 22:12:00.109349 16290 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20260811 22:12:00.114794 16309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:00.116605 16290 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:12:00.116809 16290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "beda17177b6445208b7c63b4f5b56356"
format_stamp: "Formatted at 2026-08-11 22:12:00 on dist-test-slave-68xk"
server_key: "3738044cbd96580eff4937dfd0a57590"
server_key_iv: "d82f744ccb26d6055d75abb4cef3e922"
server_key_version: "encryptionkey@0"
I20260811 22:12:00.117072 16290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:00.152223 16290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:00.159518 16290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:00.159993 16290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:00.161904 16290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:00.164418 16290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:00.164548 16290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:00.164745 16290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:00.164839 16290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:00.200836 16290 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:38745
I20260811 22:12:00.200872 16493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:38745 every 8 connection(s)
I20260811 22:12:00.202200 16290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:00.203903 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 16290
I20260811 22:12:00.204089 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:00.204571 10934 external_mini_cluster.cc:1428] Setting key 1d122e6697bc7224d5631df5fa8f5fba
Aug 11 22:12:00 dist-test-slave-68xk krb5kdc[15731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486320, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:00.222949 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:00.205086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:51945 (local address 127.10.173.190:36311)
0811 22:12:00.205620 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:00.205627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:00.208521 (+ 2894us) server_negotiation.cc:407] Connection header received
0811 22:12:00.208622 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:00.208627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:00.208709 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:00.208797 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:00.210330 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.211228 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.212714 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.213156 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.213256 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:00.216705 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:00.216709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:00.216713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:00.216746 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:00.219746 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:00.220574 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:00.220582 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:00.220588 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:00.220715 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:00.221145 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:00.221151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:00.221154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:00.221428 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:00.221598 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:00.222279 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 22:12:00.222543 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":243,"threads_started":1}
I20260811 22:12:00.223910 16498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:00.205649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36311 (local address 127.10.173.131:51945)
0811 22:12:00.206531 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:00.206578 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:00.208260 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:00.209025 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:00.209033 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:00.209560 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:00.210146 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:00.210164 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.211367 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.211371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:00.212571 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:00.212583 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.213369 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.213374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:00.213573 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:00.214301 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:00.214327 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:00.216508 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:00.219888 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:00.219897 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:00.219912 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:00.220456 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:00.220827 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:00.220832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:00.220836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:00.221031 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:00.221566 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:00.221574 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:00.221913 (+ 339us) client_negotiation.cc:769] Sending connection context
0811 22:12:00.222175 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 22:12:00.222737 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":364,"threads_started":1}
I20260811 22:12:00.225242 16494 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36311
I20260811 22:12:00.225575 16494 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:00.226349 16494 heartbeater.cc:511] Master 127.10.173.190:36311 requested a full tablet report, sending...
I20260811 22:12:00.227811 15799 ts_manager.cc:194] Registered new tserver with Master: beda17177b6445208b7c63b4f5b56356 (127.10.173.131:38745)
I20260811 22:12:00.228628 15799 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:51945
I20260811 22:12:00.232532 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:12:00 dist-test-slave-68xk krb5kdc[15731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486320, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 22:12:00 dist-test-slave-68xk krb5kdc[15731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486320, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:00.287920 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:00.273408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.10.173.190:36311)
0811 22:12:00.273660 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:00.273668 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:00.273979 (+ 311us) server_negotiation.cc:407] Connection header received
0811 22:12:00.274145 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:00.274149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:00.274233 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:00.274377 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:00.275931 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.276658 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.278040 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.278513 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.278613 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:00.282462 (+ 3849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:00.282473 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:00.282479 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:00.282523 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:00.285468 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:00.286085 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:00.286090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:00.286092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:00.286222 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:00.286671 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:00.286675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:00.286677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:00.287002 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:00.287196 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:00.287430 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 22:12:00.287590 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 22:12:00.288443 15799 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52098
I20260811 22:12:00.290432 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 15893
Aug 11 22:12:00 dist-test-slave-68xk krb5kdc[15731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486320, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:12:00.306295 16510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:00.291580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52000 (local address 127.10.173.129:44087)
0811 22:12:00.292195 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:00.292202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:00.292226 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:12:00.292307 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:00.292313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:00.292517 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:00.292676 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:00.293826 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.294925 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.296127 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.296686 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.296834 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:00.300799 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:00.300820 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:00.300840 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:00.300891 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:00.303990 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:00.304592 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:00.304599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:00.304601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:00.304736 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:00.305053 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:00.305056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:00.305058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:00.305391 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:00.305618 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:00.305817 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 22:12:00.306009 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":268,"threads_started":1}
W20260811 22:12:00.307569 16088 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52000
W20260811 22:12:00.308709 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 15893: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:12:00.308965 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 15893
I20260811 22:12:00.331916 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 16101
Aug 11 22:12:00 dist-test-slave-68xk krb5kdc[15731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486320, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:12:00.347676 16513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:00.332971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34758 (local address 127.10.173.130:43677)
0811 22:12:00.333564 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:00.333571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:00.333599 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:12:00.333661 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:00.333667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:00.333887 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:00.334025 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:00.335326 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.336358 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.337563 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.338102 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.338217 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:00.342451 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:00.342470 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:00.342486 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:00.342524 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:00.345207 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:00.345822 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:00.345829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:00.345831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:00.345956 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:00.346262 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:00.346264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:00.346266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:00.346679 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:00.346918 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:00.347174 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 22:12:00.347383 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":319,"threads_started":1}
W20260811 22:12:00.349136 16277 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34758
W20260811 22:12:00.350211 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 16101: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:12:00.350386 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 16101
I20260811 22:12:00.374858 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 16290
Aug 11 22:12:00 dist-test-slave-68xk krb5kdc[15731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486320, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:12:00.391734 16515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:00.375908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.10.173.131:38745)
0811 22:12:00.376601 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:00.376609 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:00.376639 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:12:00.376703 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:00.376726 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:00.376940 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:00.377076 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:00.378554 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.379594 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.380860 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:00.381402 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:00.381503 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:00.385903 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:00.385916 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:00.385931 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:00.385970 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:00.388906 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:00.389520 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:00.389528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:00.389530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:00.389675 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:00.390089 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:00.390092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:00.390094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:00.390595 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:00.390853 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:00.391099 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 22:12:00.391308 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":360,"threads_started":1}
W20260811 22:12:00.393021 16486 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45752
W20260811 22:12:00.394125 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 16290: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:12:00.394290 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 16290
I20260811 22:12:00.416679 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 15749
I20260811 22:12:00.417910 15844 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:00.524466 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 15749
2026-08-11T22:12:00Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4107 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:00 dist-test-slave-68xk krb5kdc[16521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:00 dist-test-slave-68xk krb5kdc[16521](info): set up 2 sockets
Aug 11 22:12:00 dist-test-slave-68xk krb5kdc[16521](info): commencing operation
krb5kdc: starting...
W20260811 22:12:01.848203 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:12:02 dist-test-slave-68xk krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:12:02.082922 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:36185
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:36307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:36185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:02.419317 16545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:02.419700 16545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:02.419862 16545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:02.427300 16545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:02.427459 16545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:02.427573 16545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:02.427685 16545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:02.427762 16545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:02.437899 16545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:36307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:36185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:36185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:02.439358 16545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:02.441293 16545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:02.453890 16554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:02.453863 16556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:02.453965 16562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:02.454766 16545 server_base.cc:1034] running on GCE node
I20260811 22:12:02.455497 16545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:02.456894 16545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:02.458148 16545 hybrid_clock.cc:648] HybridClock initialized: now 1786486322458101 us; error 67 us; skew 500 ppm
I20260811 22:12:02.458606 16545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:02 dist-test-slave-68xk krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486322, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:02.461789 16545 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:12:02.463447 16545 webserver.cc:460] Webserver started at http://127.10.173.190:43097/ using document root <none> and password file <none>
I20260811 22:12:02.464164 16545 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:02.464315 16545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:02.464651 16545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:02.467741 16545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "5541b0246d7a4214ba3c7ddc295b54eb"
format_stamp: "Formatted at 2026-08-11 22:12:02 on dist-test-slave-68xk"
server_key: "f4a076da35772c00886317992b9a7780"
server_key_iv: "43a7440a0880df15c72219046fdd93fa"
server_key_version: "encryptionkey@0"
I20260811 22:12:02.468537 16545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "5541b0246d7a4214ba3c7ddc295b54eb"
format_stamp: "Formatted at 2026-08-11 22:12:02 on dist-test-slave-68xk"
server_key: "f4a076da35772c00886317992b9a7780"
server_key_iv: "43a7440a0880df15c72219046fdd93fa"
server_key_version: "encryptionkey@0"
I20260811 22:12:02.473722 16545 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 22:12:02.477262 16574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:02.478758 16545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:02.478945 16545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "5541b0246d7a4214ba3c7ddc295b54eb"
format_stamp: "Formatted at 2026-08-11 22:12:02 on dist-test-slave-68xk"
server_key: "f4a076da35772c00886317992b9a7780"
server_key_iv: "43a7440a0880df15c72219046fdd93fa"
server_key_version: "encryptionkey@0"
I20260811 22:12:02.479125 16545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:02.495530 16545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:02.499907 16545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:02.500164 16545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:02.518807 16545 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:36185
I20260811 22:12:02.518827 16647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:36185 every 8 connection(s)
I20260811 22:12:02.520287 16545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:02.524644 16648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:02.525096 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 16545
I20260811 22:12:02.525345 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:02.526015 10934 external_mini_cluster.cc:1428] Setting key de8a5cf01f5d062aa2493db301b05daa
I20260811 22:12:02.539598 16648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb: Bootstrap starting.
Aug 11 22:12:02 dist-test-slave-68xk krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:02.543087 16648 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:02.544559 16648 log.cc:824] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:02.548189 16648 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb: No bootstrap required, opened a new log
I20260811 22:12:02.552255 16652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:02.528371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35824 (local address 127.10.173.190:36185)
0811 22:12:02.529177 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:02.529193 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:02.529244 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:12:02.531605 (+ 2361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:02.531635 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:02.532173 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:02.532652 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:02.534132 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:02.536237 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:02.537594 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:02.538708 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:02.538890 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:02.543250 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:02.543262 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:02.543285 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:02.543334 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:02.547708 (+ 4374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:02.548488 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:02.548497 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:02.548505 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:02.548743 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:02.549120 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:02.549125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:02.549128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:02.549882 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:02.550219 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:02.550738 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 22:12:02.551162 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":338,"threads_started":1}
I20260811 22:12:02.557802 16648 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5541b0246d7a4214ba3c7ddc295b54eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36185 } }
I20260811 22:12:02.558177 16648 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:02.558281 16648 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5541b0246d7a4214ba3c7ddc295b54eb, State: Initialized, Role: FOLLOWER
I20260811 22:12:02.558931 16648 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5541b0246d7a4214ba3c7ddc295b54eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36185 } }
I20260811 22:12:02.559188 16648 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:02.559334 16648 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:02.559516 16648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:02.562260 16648 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5541b0246d7a4214ba3c7ddc295b54eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36185 } }
I20260811 22:12:02.562853 16648 leader_election.cc:302] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5541b0246d7a4214ba3c7ddc295b54eb; no voters:
I20260811 22:12:02.563469 16648 leader_election.cc:288] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:02.563663 16654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:02.564476 16654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [term 1 LEADER]: Becoming Leader. State: Replica: 5541b0246d7a4214ba3c7ddc295b54eb, State: Running, Role: LEADER
I20260811 22:12:02.565056 16654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5541b0246d7a4214ba3c7ddc295b54eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36185 } }
I20260811 22:12:02.565799 16648 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:02.568528 16655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5541b0246d7a4214ba3c7ddc295b54eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5541b0246d7a4214ba3c7ddc295b54eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36185 } } }
I20260811 22:12:02.568497 16656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5541b0246d7a4214ba3c7ddc295b54eb. Latest consensus state: current_term: 1 leader_uuid: "5541b0246d7a4214ba3c7ddc295b54eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5541b0246d7a4214ba3c7ddc295b54eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36185 } } }
I20260811 22:12:02.568940 16655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:02.568979 16656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:02.570461 16667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:02.575675 16667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:02.585364 16667 catalog_manager.cc:1347] Generated new cluster ID: f1f915ef9ba9469eb474592911ece351
I20260811 22:12:02.585500 16667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:02.607836 16667 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:02.609790 16667 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:02.617851 16667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5541b0246d7a4214ba3c7ddc295b54eb: Generated new TSK 0
I20260811 22:12:02.618816 16667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:12:02.697800 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36185
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:36307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:03.046873 16682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:03.047235 16682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:03.047415 16682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:03.054950 16682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:03.055109 16682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:03.055287 16682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:12:03.065235 16682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:36307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:03.066776 16682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:03.068167 16682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:03.082361 16694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:03.082326 16697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:03.082908 16700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:03.083626 16682 server_base.cc:1034] running on GCE node
I20260811 22:12:03.084272 16682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:03.085286 16682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:03.086541 16682 hybrid_clock.cc:648] HybridClock initialized: now 1786486323086489 us; error 39 us; skew 500 ppm
I20260811 22:12:03.086897 16682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:03 dist-test-slave-68xk krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486323, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:03.090232 16682 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:12:03.091849 16682 webserver.cc:460] Webserver started at http://127.10.173.129:38335/ using document root <none> and password file <none>
I20260811 22:12:03.092631 16682 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:03.092779 16682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:03.093101 16682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:03.096134 16682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "e581624d018447fda8ad2fdd35f87893"
format_stamp: "Formatted at 2026-08-11 22:12:03 on dist-test-slave-68xk"
server_key: "4eedd07fbdf4401dca98722c28c82b9d"
server_key_iv: "1467062d27a6d37fd8cb3d95b87dcdd6"
server_key_version: "encryptionkey@0"
I20260811 22:12:03.096977 16682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "e581624d018447fda8ad2fdd35f87893"
format_stamp: "Formatted at 2026-08-11 22:12:03 on dist-test-slave-68xk"
server_key: "4eedd07fbdf4401dca98722c28c82b9d"
server_key_iv: "1467062d27a6d37fd8cb3d95b87dcdd6"
server_key_version: "encryptionkey@0"
I20260811 22:12:03.102315 16682 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:12:03.106091 16712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:03.107534 16682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:12:03.107723 16682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "e581624d018447fda8ad2fdd35f87893"
format_stamp: "Formatted at 2026-08-11 22:12:03 on dist-test-slave-68xk"
server_key: "4eedd07fbdf4401dca98722c28c82b9d"
server_key_iv: "1467062d27a6d37fd8cb3d95b87dcdd6"
server_key_version: "encryptionkey@0"
I20260811 22:12:03.107935 16682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:03.119680 16682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:03.123754 16682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:03.124049 16682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:03.125073 16682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:03.126456 16682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:03.126572 16682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:03.126729 16682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:03.126806 16682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:03.161481 16682 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:45423
I20260811 22:12:03.161522 16885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:45423 every 8 connection(s)
I20260811 22:12:03.162880 16682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:03.164170 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 16682
I20260811 22:12:03.164434 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:12:03.165396 10934 external_mini_cluster.cc:1428] Setting key 64c7fa5597de6a37e0b2580602e201b7
Aug 11 22:12:03 dist-test-slave-68xk krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486323, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:03.186134 16889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:03.166170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:37401 (local address 127.10.173.190:36185)
0811 22:12:03.166693 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:03.166697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:03.167935 (+ 1238us) server_negotiation.cc:407] Connection header received
0811 22:12:03.169826 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:03.169835 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:03.169936 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:03.170060 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:03.171653 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:03.172869 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:03.174430 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:03.175184 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:03.175330 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:03.179758 (+ 4428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:03.179763 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:03.179766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:03.179800 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:03.182810 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:03.183634 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:03.183639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:03.183643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:03.183775 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:03.184294 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:03.184299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:03.184303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:03.184590 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:03.184748 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:03.185528 (+ 780us) server_negotiation.cc:299] Negotiation successful
0811 22:12:03.185772 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":285,"threads_started":1}
I20260811 22:12:03.187206 16890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:03.166871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36185 (local address 127.10.173.129:37401)
0811 22:12:03.167760 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:03.167806 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:03.169548 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:03.170254 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:03.170261 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:03.170878 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:03.171442 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:03.171459 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:03.173042 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:03.173046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:03.174256 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:03.174273 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:03.175400 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:03.175406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:03.175598 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:03.176514 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:03.176548 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:03.179500 (+ 2952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:03.182981 (+ 3481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:03.182991 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:03.183009 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:03.183493 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:03.183951 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:03.183956 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:03.183959 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:03.184163 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:03.184726 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:03.184735 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:03.185107 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 22:12:03.185400 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 22:12:03.185948 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":360,"threads_started":1}
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
I20260811 22:12:03.188898 16886 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36185
I20260811 22:12:03.189229 16886 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:03.190011 16886 heartbeater.cc:511] Master 127.10.173.190:36185 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
I20260811 22:12:03.192480 16600 ts_manager.cc:194] Registered new tserver with Master: e581624d018447fda8ad2fdd35f87893 (127.10.173.129:45423)
I20260811 22:12:03.193950 16600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:37401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:12:03.219818 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36185
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:36307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:03.561403 16894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:03.561684 16894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:03.561846 16894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:03.569111 16894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:03.569235 16894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:03.569419 16894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:12:03.579106 16894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:36307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:03.580461 16894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:03.581810 16894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:03.596187 16904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:03.596021 16902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:03.595953 16906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:03.597194 16894 server_base.cc:1034] running on GCE node
I20260811 22:12:03.597806 16894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:03.598834 16894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:03.600100 16894 hybrid_clock.cc:648] HybridClock initialized: now 1786486323600000 us; error 40 us; skew 500 ppm
I20260811 22:12:03.600425 16894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:03 dist-test-slave-68xk krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486323, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:03.608196 16894 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:12:03.609673 16894 webserver.cc:460] Webserver started at http://127.10.173.130:45351/ using document root <none> and password file <none>
I20260811 22:12:03.610484 16894 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:03.610642 16894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:03.611053 16894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:03.614166 16894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "5854db4fe0564d049013eb5dca6b74cd"
format_stamp: "Formatted at 2026-08-11 22:12:03 on dist-test-slave-68xk"
server_key: "fd745011306eb549fea88ad4aa6516ec"
server_key_iv: "3e2223351813088e9b86ed6fc025da3d"
server_key_version: "encryptionkey@0"
I20260811 22:12:03.615080 16894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "5854db4fe0564d049013eb5dca6b74cd"
format_stamp: "Formatted at 2026-08-11 22:12:03 on dist-test-slave-68xk"
server_key: "fd745011306eb549fea88ad4aa6516ec"
server_key_iv: "3e2223351813088e9b86ed6fc025da3d"
server_key_version: "encryptionkey@0"
I20260811 22:12:03.620745 16894 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 22:12:03.625099 16916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:03.626830 16894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:03.627025 16894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "5854db4fe0564d049013eb5dca6b74cd"
format_stamp: "Formatted at 2026-08-11 22:12:03 on dist-test-slave-68xk"
server_key: "fd745011306eb549fea88ad4aa6516ec"
server_key_iv: "3e2223351813088e9b86ed6fc025da3d"
server_key_version: "encryptionkey@0"
I20260811 22:12:03.627310 16894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:03.658151 16894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:03.662879 16894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:03.663204 16894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:03.664227 16894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:03.665732 16894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:03.665833 16894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:03.665993 16894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:03.666077 16894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:03.701988 16894 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:45553
I20260811 22:12:03.702047 17096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:45553 every 8 connection(s)
I20260811 22:12:03.703483 16894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:03.713155 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 16894
I20260811 22:12:03.713390 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:12:03.713852 10934 external_mini_cluster.cc:1428] Setting key d75e7a3b1a449f63d482a0fe804f3cc6
Aug 11 22:12:03 dist-test-slave-68xk krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486323, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:03.724330 17100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:03.706144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:48803 (local address 127.10.173.190:36185)
0811 22:12:03.706638 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:03.706643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:03.707588 (+ 945us) server_negotiation.cc:407] Connection header received
0811 22:12:03.709455 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:03.709461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:03.709565 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:03.709690 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:03.711304 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:03.712089 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:03.713805 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:03.714289 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:03.714455 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:03.718576 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:03.718582 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:03.718586 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:03.718618 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:03.721212 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:03.721941 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:03.721947 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:03.721957 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:03.722083 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:03.722561 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:03.722570 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:03.722572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:03.722830 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:03.722989 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:03.723706 (+ 717us) server_negotiation.cc:299] Negotiation successful
0811 22:12:03.723932 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":277,"threads_started":1}
I20260811 22:12:03.725423 17101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:03.706635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36185 (local address 127.10.173.130:48803)
0811 22:12:03.707381 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:03.707427 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:03.709176 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:03.709929 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:03.709937 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:03.710559 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:03.711130 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:03.711148 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:03.712253 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:03.712257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:03.713663 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:03.713676 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:03.714557 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:03.714566 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:03.714857 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:03.715695 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:03.715741 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:03.718310 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:03.721361 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:03.721370 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:03.721386 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:03.721809 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:03.722199 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:03.722204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:03.722207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:03.722393 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:03.722964 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:03.722972 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:03.723336 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 22:12:03.723608 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 22:12:03.724161 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":322,"threads_started":1}
I20260811 22:12:03.727121 17097 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36185
I20260811 22:12:03.727444 17097 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:03.728181 17097 heartbeater.cc:511] Master 127.10.173.190:36185 requested a full tablet report, sending...
I20260811 22:12:03.729777 16600 ts_manager.cc:194] Registered new tserver with Master: 5854db4fe0564d049013eb5dca6b74cd (127.10.173.130:45553)
I20260811 22:12:03.731313 16600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:48803
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:12:03.769635 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36185
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:36307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:04.109627 17106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:04.109937 17106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:04.110037 17106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:04.117475 17106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:04.117583 17106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:04.117697 17106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:12:04.127712 17106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:36307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:04.128908 17106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:04.130148 17106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:04.143764 17114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:04.143877 17113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:04.143847 17118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:04.144665 17106 server_base.cc:1034] running on GCE node
I20260811 22:12:04.145349 17106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:04.146338 17106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:04.147599 17106 hybrid_clock.cc:648] HybridClock initialized: now 1786486324147544 us; error 53 us; skew 500 ppm
I20260811 22:12:04.147925 17106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:04 dist-test-slave-68xk krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486324, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:04.151360 17106 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:12:04.152869 17106 webserver.cc:460] Webserver started at http://127.10.173.131:42277/ using document root <none> and password file <none>
I20260811 22:12:04.153571 17106 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:04.153723 17106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:04.154045 17106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:04.156955 17106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "18d5f43dae04447b8119a364fc44f6b4"
format_stamp: "Formatted at 2026-08-11 22:12:04 on dist-test-slave-68xk"
server_key: "fe9ecb23b0c5bef9751f83754cce9850"
server_key_iv: "4dd826757439015218eefc4e6ca28d76"
server_key_version: "encryptionkey@0"
I20260811 22:12:04.157773 17106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "18d5f43dae04447b8119a364fc44f6b4"
format_stamp: "Formatted at 2026-08-11 22:12:04 on dist-test-slave-68xk"
server_key: "fe9ecb23b0c5bef9751f83754cce9850"
server_key_iv: "4dd826757439015218eefc4e6ca28d76"
server_key_version: "encryptionkey@0"
I20260811 22:12:04.163113 17106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 22:12:04.166927 17129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:04.168357 17106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:12:04.168540 17106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "18d5f43dae04447b8119a364fc44f6b4"
format_stamp: "Formatted at 2026-08-11 22:12:04 on dist-test-slave-68xk"
server_key: "fe9ecb23b0c5bef9751f83754cce9850"
server_key_iv: "4dd826757439015218eefc4e6ca28d76"
server_key_version: "encryptionkey@0"
I20260811 22:12:04.168752 17106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:04.184957 17106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:04.189463 17106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:04.189764 17106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:04.190892 17106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:04.192327 17106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:04.192432 17106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:04.192587 17106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:04.192660 17106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:04.197697 16886 heartbeater.cc:503] Master 127.10.173.190:36185 was elected leader, sending a full tablet report...
I20260811 22:12:04.228741 17106 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:42699
I20260811 22:12:04.228834 17317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:42699 every 8 connection(s)
I20260811 22:12:04.230041 17106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:04.232959 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 17106
I20260811 22:12:04.233233 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:04.233731 10934 external_mini_cluster.cc:1428] Setting key d4b4e1099aef94d35f35a95f66e4b27a
Aug 11 22:12:04 dist-test-slave-68xk krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486324, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:04.251958 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:04.232922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:35333 (local address 127.10.173.190:36185)
0811 22:12:04.234987 (+ 2065us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:04.234994 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:04.235016 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:12:04.237213 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:04.237237 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:04.237339 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:04.237465 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:04.239065 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.239840 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.241496 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.241893 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.241986 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:04.245647 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:04.245653 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:04.245656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:04.245693 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:04.248748 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.249639 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.249644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.249654 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.249822 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.250279 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.250282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.250284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.250601 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:04.250729 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:04.251368 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 22:12:04.251585 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1933,"thread_start_us":1809,"threads_started":1}
I20260811 22:12:04.252797 17321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:04.233589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36185 (local address 127.10.173.131:35333)
0811 22:12:04.234688 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:04.234760 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:04.236871 (+ 2111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:04.237699 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:04.237707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:04.238234 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:04.238913 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:04.238931 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.239983 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.239987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:04.241364 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:04.241377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.242102 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.242110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:04.242336 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:04.243067 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:04.243098 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:04.245426 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:04.248897 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:04.248903 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:04.248915 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:04.249504 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:04.249949 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:04.249954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:04.249957 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:04.250172 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:04.250713 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:04.250719 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:04.251028 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 22:12:04.251265 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 22:12:04.251677 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":391,"threads_started":1}
I20260811 22:12:04.254105 17318 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36185
I20260811 22:12:04.254467 17318 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:04.255247 17318 heartbeater.cc:511] Master 127.10.173.190:36185 requested a full tablet report, sending...
I20260811 22:12:04.256662 16600 ts_manager.cc:194] Registered new tserver with Master: 18d5f43dae04447b8119a364fc44f6b4 (127.10.173.131:42699)
I20260811 22:12:04.257493 16600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:35333
I20260811 22:12:04.262342 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:12:04 dist-test-slave-68xk krb5kdc[16521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:12:04 dist-test-slave-68xk krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:12:04.320092 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:04.303080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33836 (local address 127.10.173.129:45423)
0811 22:12:04.303840 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:04.303850 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:04.303881 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:12:04.303952 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:04.303959 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:04.304192 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:04.304369 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:04.305888 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.306995 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.308278 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.308845 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.308983 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:04.313593 (+ 4610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:04.313607 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:04.313622 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:04.313685 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:04.316889 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.317601 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.317610 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.317612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.317763 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.318218 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.318223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.318225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.318834 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:04.319100 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:04.319435 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 22:12:04.319739 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":374,"threads_started":1}
Aug 11 22:12:04 dist-test-slave-68xk krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:04.339016 17320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:04.324786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35836 (local address 127.10.173.190:36185)
0811 22:12:04.325010 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:04.325017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:04.325166 (+ 149us) server_negotiation.cc:407] Connection header received
0811 22:12:04.325350 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:04.325362 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:04.325430 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:04.325538 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:04.326970 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.327704 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.328944 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.329370 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.329515 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:04.333545 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:04.333551 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:04.333554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:04.333589 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:04.336307 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.336945 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.336949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.336951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.337067 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.337579 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.337583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.337585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.337918 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:04.338108 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:04.338475 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 22:12:04.338644 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 22:12:04.339613 16600 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35836
I20260811 22:12:04.340636 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 16682
I20260811 22:12:04.352562 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:04.341543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.10.173.129:45423)
0811 22:12:04.341701 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:04.341705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:04.341990 (+ 285us) server_negotiation.cc:407] Connection header received
0811 22:12:04.342114 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:04.342118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:04.342180 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:04.342281 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:04.343459 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.344116 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.345322 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.345722 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.345820 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:04.347986 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:04.347990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:04.347994 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:04.348023 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:04.350458 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.351067 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.351071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.351074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.351191 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.351509 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.351512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.351514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.351740 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:04.351913 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:04.352152 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 22:12:04.352331 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:12:04.353188 16880 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:04.435422 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 16682
I20260811 22:12:04.462821 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 16894
Aug 11 22:12:04 dist-test-slave-68xk krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:12:04.479642 17336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:04.463836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.10.173.130:45553)
0811 22:12:04.464400 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:04.464408 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:04.464434 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:12:04.464502 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:04.464508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:04.464722 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:04.464905 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:04.466225 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.467430 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.468696 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.469238 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.469384 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:04.473979 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:04.473990 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:04.474004 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:04.474042 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:04.476831 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.477483 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.477491 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.477493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.477620 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.478000 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.478005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.478009 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.478351 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:04.478699 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:04.479024 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 22:12:04.479290 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":290,"threads_started":1}
I20260811 22:12:04.480962 17092 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:04.563875 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 16894
I20260811 22:12:04.589910 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 17106
Aug 11 22:12:04 dist-test-slave-68xk krb5kdc[16521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:12:04.607348 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:04.590957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38394 (local address 127.10.173.131:42699)
0811 22:12:04.591584 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:04.591591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:04.591621 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:12:04.591684 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:04.591690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:04.591909 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:04.592092 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:04.593446 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.594642 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.595948 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:04.596533 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:04.596692 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:04.601167 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:04.601179 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:04.601195 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:04.601233 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:04.604295 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.605071 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.605078 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.605081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.605206 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:04.605593 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:04.605599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:04.605602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:04.605966 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:04.606320 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:04.606691 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 22:12:04.606984 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":304,"threads_started":1}
I20260811 22:12:04.608644 17312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:04.693317 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 17106
I20260811 22:12:04.718186 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 16545
I20260811 22:12:04.719484 16642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:04.824395 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 16545
2026-08-11T22:12:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4297 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:04 dist-test-slave-68xk krb5kdc[17344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:04 dist-test-slave-68xk krb5kdc[17344](info): set up 2 sockets
Aug 11 22:12:04 dist-test-slave-68xk krb5kdc[17344](info): commencing operation
krb5kdc: starting...
W20260811 22:12:06.151069 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:12:06 dist-test-slave-68xk krb5kdc[17344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:12:06.386612 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41089
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:41089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:06.726792 17366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:06.727097 17366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:06.727309 17366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:06.734583 17366 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:06.734702 17366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:06.734791 17366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:06.734886 17366 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:06.734972 17366 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:06.744747 17366 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:41089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:06.746048 17366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:06.747438 17366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:06.759123 17377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:06.759119 17373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:06.759171 17374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:06.760248 17366 server_base.cc:1034] running on GCE node
I20260811 22:12:06.761052 17366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:06.762529 17366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:06.763792 17366 hybrid_clock.cc:648] HybridClock initialized: now 1786486326763743 us; error 46 us; skew 500 ppm
I20260811 22:12:06.764205 17366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:06 dist-test-slave-68xk krb5kdc[17344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486326, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:06.767514 17366 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:12:06.768815 17366 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:06.768951 17366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:06.769289 17366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:06.772307 17366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "faa6b7883ce848829ced36ef5d0dd300"
format_stamp: "Formatted at 2026-08-11 22:12:06 on dist-test-slave-68xk"
server_key: "0f1281076af0344fc55c1e5003697b39"
server_key_iv: "89c6dd614133c7a2346a5b9c25968d46"
server_key_version: "encryptionkey@0"
I20260811 22:12:06.773165 17366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "faa6b7883ce848829ced36ef5d0dd300"
format_stamp: "Formatted at 2026-08-11 22:12:06 on dist-test-slave-68xk"
server_key: "0f1281076af0344fc55c1e5003697b39"
server_key_iv: "89c6dd614133c7a2346a5b9c25968d46"
server_key_version: "encryptionkey@0"
I20260811 22:12:06.778067 17366 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 22:12:06.781651 17385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:06.782963 17366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:06.783180 17366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "faa6b7883ce848829ced36ef5d0dd300"
format_stamp: "Formatted at 2026-08-11 22:12:06 on dist-test-slave-68xk"
server_key: "0f1281076af0344fc55c1e5003697b39"
server_key_iv: "89c6dd614133c7a2346a5b9c25968d46"
server_key_version: "encryptionkey@0"
I20260811 22:12:06.783423 17366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:06.798514 17366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:06.802745 17366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:06.803038 17366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:06.821918 17366 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:41089
I20260811 22:12:06.821939 17463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:41089 every 8 connection(s)
I20260811 22:12:06.822902 17366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:06.826725 17464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:06.829746 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 17366
I20260811 22:12:06.830011 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:06.830741 10934 external_mini_cluster.cc:1428] Setting key 2538ab2d40da1e65ef76347a29435113
I20260811 22:12:06.840298 17464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300: Bootstrap starting.
I20260811 22:12:06.843998 17464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:06.845098 17464 log.cc:824] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:12:06 dist-test-slave-68xk krb5kdc[17344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:06.848501 17464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300: No bootstrap required, opened a new log
I20260811 22:12:06.857700 17468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:06.833021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58284 (local address 127.10.173.190:41089)
0811 22:12:06.833920 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:06.833935 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:06.833987 (+ 52us) server_negotiation.cc:407] Connection header received
0811 22:12:06.835992 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:06.836029 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:06.836548 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:06.837025 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:06.838611 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:06.840563 (+ 1952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:06.841995 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:06.843069 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:06.843253 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:06.848078 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:06.848090 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:06.848110 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:06.848165 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:06.852594 (+ 4429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:06.853432 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:06.853441 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:06.853450 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:06.853667 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:06.854062 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:06.854067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:06.854071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:06.854931 (+ 860us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:06.855270 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:06.855702 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 22:12:06.856197 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":374,"threads_started":1}
I20260811 22:12:06.859707 17464 raft_consensus.cc:348] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa6b7883ce848829ced36ef5d0dd300" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41089 } }
I20260811 22:12:06.860169 17464 raft_consensus.cc:374] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:06.860345 17464 raft_consensus.cc:729] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa6b7883ce848829ced36ef5d0dd300, State: Initialized, Role: FOLLOWER
I20260811 22:12:06.861055 17464 consensus_queue.cc:260] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa6b7883ce848829ced36ef5d0dd300" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41089 } }
I20260811 22:12:06.861339 17464 raft_consensus.cc:388] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:06.861505 17464 raft_consensus.cc:482] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:06.861689 17464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:06.864063 17464 raft_consensus.cc:504] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa6b7883ce848829ced36ef5d0dd300" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41089 } }
I20260811 22:12:06.864607 17464 leader_election.cc:302] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faa6b7883ce848829ced36ef5d0dd300; no voters:
I20260811 22:12:06.865351 17464 leader_election.cc:288] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:06.865521 17470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:06.866308 17470 raft_consensus.cc:686] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [term 1 LEADER]: Becoming Leader. State: Replica: faa6b7883ce848829ced36ef5d0dd300, State: Running, Role: LEADER
I20260811 22:12:06.867033 17470 consensus_queue.cc:237] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa6b7883ce848829ced36ef5d0dd300" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41089 } }
I20260811 22:12:06.867851 17464 sys_catalog.cc:564] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:06.870671 17472 sys_catalog.cc:455] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faa6b7883ce848829ced36ef5d0dd300" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa6b7883ce848829ced36ef5d0dd300" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41089 } } }
I20260811 22:12:06.870658 17473 sys_catalog.cc:455] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faa6b7883ce848829ced36ef5d0dd300. Latest consensus state: current_term: 1 leader_uuid: "faa6b7883ce848829ced36ef5d0dd300" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faa6b7883ce848829ced36ef5d0dd300" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41089 } } }
I20260811 22:12:06.871129 17472 sys_catalog.cc:458] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:06.871215 17473 sys_catalog.cc:458] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:06.872836 17479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:06.878561 17479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:06.888232 17479 catalog_manager.cc:1347] Generated new cluster ID: e44664add60d4b02a700202c3c2e34ae
I20260811 22:12:06.888367 17479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:06.900898 17479 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:06.902225 17479 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:06.908661 17479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300: Generated new TSK 0
I20260811 22:12:06.909669 17479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:12:06.968155 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41089
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:07.318081 17500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:07.318384 17500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:07.318635 17500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:07.326004 17500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:07.326112 17500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:07.326315 17500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:12:07.336251 17500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:07.337615 17500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:07.339114 17500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:07.352628 17508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:07.352537 17506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:07.352594 17510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:07.353595 17500 server_base.cc:1034] running on GCE node
I20260811 22:12:07.354310 17500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:07.355556 17500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:07.356814 17500 hybrid_clock.cc:648] HybridClock initialized: now 1786486327356730 us; error 73 us; skew 500 ppm
I20260811 22:12:07.357205 17500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:07 dist-test-slave-68xk krb5kdc[17344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486327, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:07.360972 17500 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:12:07.362510 17500 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:07.362655 17500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:07.363054 17500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:07.366485 17500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "4e7c727860ae41f3af7dcfa17cab599c"
format_stamp: "Formatted at 2026-08-11 22:12:07 on dist-test-slave-68xk"
server_key: "74785a08be7ba4020fdf0fafc8482ceb"
server_key_iv: "3647de44b2cbb76bdc88a65fd3a9ae4c"
server_key_version: "encryptionkey@0"
I20260811 22:12:07.367471 17500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "4e7c727860ae41f3af7dcfa17cab599c"
format_stamp: "Formatted at 2026-08-11 22:12:07 on dist-test-slave-68xk"
server_key: "74785a08be7ba4020fdf0fafc8482ceb"
server_key_iv: "3647de44b2cbb76bdc88a65fd3a9ae4c"
server_key_version: "encryptionkey@0"
I20260811 22:12:07.373494 17500 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260811 22:12:07.377712 17517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:07.379288 17500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:07.379479 17500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "4e7c727860ae41f3af7dcfa17cab599c"
format_stamp: "Formatted at 2026-08-11 22:12:07 on dist-test-slave-68xk"
server_key: "74785a08be7ba4020fdf0fafc8482ceb"
server_key_iv: "3647de44b2cbb76bdc88a65fd3a9ae4c"
server_key_version: "encryptionkey@0"
I20260811 22:12:07.379688 17500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:07.397699 17500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:07.402199 17500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:07.402555 17500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:07.403643 17500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:07.405166 17500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:07.405256 17500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:07.405395 17500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:07.405500 17500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:07.440697 17500 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:40293
I20260811 22:12:07.440749 17692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:40293 every 8 connection(s)
I20260811 22:12:07.441558 17500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:07.451351 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 17500
I20260811 22:12:07.451618 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:12:07.452521 10934 external_mini_cluster.cc:1428] Setting key 5e52702294518e2825f52585e26206c1
Aug 11 22:12:07 dist-test-slave-68xk krb5kdc[17344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486327, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:07.466310 17696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:07.444096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:52199 (local address 127.10.173.190:41089)
0811 22:12:07.444594 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:07.444599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:07.445596 (+ 997us) server_negotiation.cc:407] Connection header received
0811 22:12:07.447847 (+ 2251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:07.447855 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:07.447960 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:07.448107 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:07.449749 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:07.450662 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:07.453472 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:07.454096 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:07.454257 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:07.459222 (+ 4965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:07.459232 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:07.459240 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:07.459320 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:07.462829 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:07.463674 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:07.463682 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:07.463686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:07.463826 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:07.464310 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:07.464317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:07.464320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:07.464613 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:07.464787 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:07.465619 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 22:12:07.465866 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":373,"thread_start_us":248,"threads_started":1}
I20260811 22:12:07.467514 17699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:07.444634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41089 (local address 127.10.173.129:52199)
0811 22:12:07.445406 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:07.445459 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:07.447548 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:07.448389 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:07.448397 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:07.448964 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:07.449557 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:07.449582 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:07.450856 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:07.450863 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:07.453291 (+ 2428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:07.453310 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:07.454350 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:07.454360 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:07.454678 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:07.455453 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:07.455488 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:07.458868 (+ 3380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:07.462987 (+ 4119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:07.462997 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:07.463015 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:07.463518 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:07.463957 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:07.463962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:07.463966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:07.464179 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:07.464759 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:07.464774 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:07.465193 (+ 419us) client_negotiation.cc:769] Sending connection context
0811 22:12:07.465493 (+ 300us) client_negotiation.cc:240] Negotiation successful
0811 22:12:07.466113 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":308,"threads_started":1}
I20260811 22:12:07.469298 17693 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41089
I20260811 22:12:07.469687 17693 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:07.470660 17693 heartbeater.cc:511] Master 127.10.173.190:41089 requested a full tablet report, sending...
I20260811 22:12:07.473058 17408 ts_manager.cc:194] Registered new tserver with Master: 4e7c727860ae41f3af7dcfa17cab599c (127.10.173.129:40293)
I20260811 22:12:07.474722 17408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:52199
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:12:07.510622 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41089
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:07.858861 17703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:07.859151 17703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:07.859315 17703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:07.866760 17703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:07.866873 17703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:07.867041 17703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:12:07.877336 17703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:07.878723 17703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:07.880074 17703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:07.895112 17709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:07.895157 17710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:07.895130 17714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:07.896186 17703 server_base.cc:1034] running on GCE node
I20260811 22:12:07.896807 17703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:07.897794 17703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:07.899027 17703 hybrid_clock.cc:648] HybridClock initialized: now 1786486327898973 us; error 53 us; skew 500 ppm
I20260811 22:12:07.899367 17703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:07 dist-test-slave-68xk krb5kdc[17344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486327, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:07.902828 17703 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:12:07.904063 17703 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:07.904210 17703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:07.904541 17703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:07.907608 17703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "6ff079bca43d4114b05767db0a598664"
format_stamp: "Formatted at 2026-08-11 22:12:07 on dist-test-slave-68xk"
server_key: "e3e1babd64410b6bfbe2e65a712a5dad"
server_key_iv: "15041e852657184d6e4f087dc49c6f82"
server_key_version: "encryptionkey@0"
I20260811 22:12:07.908447 17703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "6ff079bca43d4114b05767db0a598664"
format_stamp: "Formatted at 2026-08-11 22:12:07 on dist-test-slave-68xk"
server_key: "e3e1babd64410b6bfbe2e65a712a5dad"
server_key_iv: "15041e852657184d6e4f087dc49c6f82"
server_key_version: "encryptionkey@0"
I20260811 22:12:07.913908 17703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:12:07.917711 17725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:07.919204 17703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:07.919387 17703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "6ff079bca43d4114b05767db0a598664"
format_stamp: "Formatted at 2026-08-11 22:12:07 on dist-test-slave-68xk"
server_key: "e3e1babd64410b6bfbe2e65a712a5dad"
server_key_iv: "15041e852657184d6e4f087dc49c6f82"
server_key_version: "encryptionkey@0"
I20260811 22:12:07.919621 17703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:07.933079 17703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:07.937403 17703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:07.937698 17703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:07.938732 17703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:07.940079 17703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:07.940214 17703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:07.940353 17703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:07.940446 17703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:07.974555 17703 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:43113
I20260811 22:12:07.974620 17894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:43113 every 8 connection(s)
I20260811 22:12:07.975468 17703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:07.984720 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 17703
I20260811 22:12:07.984949 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:12:07.985558 10934 external_mini_cluster.cc:1428] Setting key c9cb90974e6b2141d1c8cc705b007787
Aug 11 22:12:07 dist-test-slave-68xk krb5kdc[17344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486327, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:07.999569 17897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:07.978075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:37909 (local address 127.10.173.190:41089)
0811 22:12:07.978580 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:07.978588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:07.979651 (+ 1063us) server_negotiation.cc:407] Connection header received
0811 22:12:07.981420 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:07.981426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:07.981494 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:07.981595 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:07.983381 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:07.984239 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:07.986599 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:07.987284 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:07.987420 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:07.991557 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:07.991566 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:07.991573 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:07.991621 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:07.995401 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:07.996467 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:07.996476 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:07.996480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:07.996648 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:07.997170 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:07.997176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:07.997179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:07.997686 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:07.997864 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:07.998696 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 22:12:07.998996 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":242,"threads_started":1}
I20260811 22:12:08.000381 17900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:07.978721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41089 (local address 127.10.173.130:37909)
0811 22:12:07.979448 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:07.979490 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:07.981139 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:07.981831 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:07.981839 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:07.982368 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:07.983135 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:07.983153 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:07.984410 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:07.984419 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:07.986404 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:07.986441 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:07.987630 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:07.987644 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:07.987935 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:07.988689 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:07.988718 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:07.991275 (+ 2557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:07.995577 (+ 4302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:07.995586 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:07.995600 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:07.996294 (+ 694us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:07.996789 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:07.996795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:07.996801 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:07.997014 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:07.997864 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:07.997870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:07.998221 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 22:12:07.998560 (+ 339us) client_negotiation.cc:240] Negotiation successful
0811 22:12:07.999002 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":301,"threads_started":1}
I20260811 22:12:08.002005 17895 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41089
I20260811 22:12:08.002352 17895 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:08.003118 17895 heartbeater.cc:511] Master 127.10.173.190:41089 requested a full tablet report, sending...
I20260811 22:12:08.004772 17408 ts_manager.cc:194] Registered new tserver with Master: 6ff079bca43d4114b05767db0a598664 (127.10.173.130:43113)
I20260811 22:12:08.006078 17408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:37909
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:12:08.053699 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41089
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:08.387938 17904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:08.388228 17904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:08.388424 17904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:08.395831 17904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:08.395941 17904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:08.396163 17904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:12:08.405947 17904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:08.407328 17904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:08.408689 17904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:08.422003 17911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:08.422067 17915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:08.422232 17917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:08.423440 17904 server_base.cc:1034] running on GCE node
I20260811 22:12:08.424177 17904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:08.425283 17904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:08.426522 17904 hybrid_clock.cc:648] HybridClock initialized: now 1786486328426485 us; error 61 us; skew 500 ppm
I20260811 22:12:08.426883 17904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:08 dist-test-slave-68xk krb5kdc[17344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486328, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:08.430086 17904 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:12:08.431468 17904 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:08.431608 17904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:08.431936 17904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:08.435041 17904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "8f0be427b355483b8685ab95db21d210"
format_stamp: "Formatted at 2026-08-11 22:12:08 on dist-test-slave-68xk"
server_key: "4eac8661b5d5ec38b5b8afe7815809cb"
server_key_iv: "3e3738e6461d064c0a441648ea968862"
server_key_version: "encryptionkey@0"
I20260811 22:12:08.435885 17904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "8f0be427b355483b8685ab95db21d210"
format_stamp: "Formatted at 2026-08-11 22:12:08 on dist-test-slave-68xk"
server_key: "4eac8661b5d5ec38b5b8afe7815809cb"
server_key_iv: "3e3738e6461d064c0a441648ea968862"
server_key_version: "encryptionkey@0"
I20260811 22:12:08.441431 17904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:12:08.445092 17925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:08.446467 17904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:12:08.446650 17904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "8f0be427b355483b8685ab95db21d210"
format_stamp: "Formatted at 2026-08-11 22:12:08 on dist-test-slave-68xk"
server_key: "4eac8661b5d5ec38b5b8afe7815809cb"
server_key_iv: "3e3738e6461d064c0a441648ea968862"
server_key_version: "encryptionkey@0"
I20260811 22:12:08.446870 17904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:08.464146 17904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:08.469036 17904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:08.469281 17904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:08.470292 17904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:08.471736 17904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:08.471827 17904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:08.471899 17904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:08.471958 17904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:08.477912 17693 heartbeater.cc:503] Master 127.10.173.190:41089 was elected leader, sending a full tablet report...
I20260811 22:12:08.508985 17904 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:36519
I20260811 22:12:08.509081 18105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:36519 every 8 connection(s)
I20260811 22:12:08.509891 17904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:08.517035 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 17904
I20260811 22:12:08.517307 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:08.517786 10934 external_mini_cluster.cc:1428] Setting key 6486ac4b9fffc6129f9285cdab7223e1
Aug 11 22:12:08 dist-test-slave-68xk krb5kdc[17344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486328, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:08.531172 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.512646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:57899 (local address 127.10.173.190:41089)
0811 22:12:08.513110 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:08.513115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:08.514172 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 22:12:08.516059 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:08.516064 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:08.516173 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:08.516287 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:08.517957 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.519069 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.520937 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.521561 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.521674 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.525322 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:08.525327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:08.525331 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:08.525367 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:08.528240 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:08.528967 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:08.528972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:08.528974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:08.529120 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:08.529514 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:08.529517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:08.529519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:08.529805 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:08.529924 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:08.530635 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 22:12:08.530826 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":219,"threads_started":1}
I20260811 22:12:08.532543 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.513254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41089 (local address 127.10.173.131:57899)
0811 22:12:08.513985 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:08.514031 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:08.515771 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:08.516495 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:08.516503 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:08.517043 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:08.517731 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.517756 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.519288 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.519295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.520734 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.520747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.521778 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.521783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.521971 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.522746 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:08.522769 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:08.525088 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:08.528380 (+ 3292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:08.528387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:08.528401 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:08.528832 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:08.529233 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:08.529236 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:08.529238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:08.529416 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:08.529914 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:08.529920 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:08.530252 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 22:12:08.530538 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 22:12:08.530949 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":306,"threads_started":1}
I20260811 22:12:08.534101 18106 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41089
I20260811 22:12:08.534478 18106 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:08.535171 18106 heartbeater.cc:511] Master 127.10.173.190:41089 requested a full tablet report, sending...
I20260811 22:12:08.536639 17407 ts_manager.cc:194] Registered new tserver with Master: 8f0be427b355483b8685ab95db21d210 (127.10.173.131:36519)
I20260811 22:12:08.537593 17407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:57899
I20260811 22:12:08.546484 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:12:08.564976 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.551469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58300 (local address 127.10.173.190:41089)
0811 22:12:08.551677 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:08.551683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:08.552323 (+ 640us) server_negotiation.cc:407] Connection header received
0811 22:12:08.552427 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:08.552431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:08.552499 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:08.552589 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:08.554117 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.555017 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.556309 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.556753 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.556858 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.559481 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:08.559486 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:08.559490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:08.559528 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:08.562652 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:08.563252 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:08.563256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:08.563259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:08.563380 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:08.563701 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:08.563705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:08.563707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:08.564055 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:08.564239 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:08.564532 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 22:12:08.564737 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:12:08.569288 17407 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58300:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:08.576385 17407 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:08.601387 18131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.588959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:43113 (local address 127.0.0.1:51440)
0811 22:12:08.590097 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:08.590139 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:08.590308 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:08.594899 (+ 4591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:08.594905 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:08.594943 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:08.595415 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.595429 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.597332 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.597340 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.598937 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.598953 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.600326 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.600333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.600548 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.600616 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 22:12:08.600735 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 22:12:08.600948 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"mutex_wait_us":21,"thread_start_us":3137,"threads_started":1}
I20260811 22:12:08.602084 18126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.589335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:36519 (local address 127.0.0.1:34586)
0811 22:12:08.592400 (+ 3065us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:08.592424 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:08.592603 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:08.593542 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:08.593552 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:08.593593 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:08.594086 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.594102 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.596223 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.596231 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.598987 (+ 2756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.599002 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.601227 (+ 2225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.601233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.601426 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.601462 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:12:08.601561 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 22:12:08.601694 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2904,"mutex_wait_us":53,"thread_start_us":390,"threads_started":1}
I20260811 22:12:08.602638 18128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.589170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.10.173.130:43113)
0811 22:12:08.594245 (+ 5075us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:08.594257 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:08.594288 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:12:08.594387 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:08.594396 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:08.594640 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:08.595551 (+ 911us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:08.595746 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.597188 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.599097 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.600056 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.601941 (+ 1885us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.602058 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:08.602112 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 22:12:08.602332 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4907,"thread_start_us":4806,"threads_started":1}
I20260811 22:12:08.602973 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.589969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34586 (local address 127.10.173.131:36519)
0811 22:12:08.592141 (+ 2172us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:08.592151 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:08.592780 (+ 629us) server_negotiation.cc:407] Connection header received
0811 22:12:08.592923 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:08.592934 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:08.593252 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:08.593500 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:08.594390 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.596040 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.599188 (+ 3148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.600658 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.600859 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.601071 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:08.602401 (+ 1330us) server_negotiation.cc:299] Negotiation successful
0811 22:12:08.602614 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1956,"thread_start_us":1751,"threads_started":1}
I20260811 22:12:08.604027 18134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.592547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40293 (local address 127.0.0.1:44288)
0811 22:12:08.595170 (+ 2623us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:08.595189 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:08.595348 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:08.596143 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:08.596147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:08.596175 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:08.596726 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.596738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.598693 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.598697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.600881 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.600897 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.603188 (+ 2291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.603195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.603378 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.603410 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:12:08.603512 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:12:08.603637 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2464,"mutex_wait_us":3,"thread_start_us":2297,"threads_started":1}
I20260811 22:12:08.604250 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.593040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44288 (local address 127.10.173.129:40293)
0811 22:12:08.594680 (+ 1640us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:08.594689 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:08.595319 (+ 630us) server_negotiation.cc:407] Connection header received
0811 22:12:08.595520 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:08.595528 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:08.595735 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:08.595903 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:08.596929 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.598500 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.601065 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.602640 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.602861 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.603036 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:08.603582 (+ 546us) server_negotiation.cc:299] Negotiation successful
0811 22:12:08.603766 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1460,"thread_start_us":1304,"threads_started":1}
I20260811 22:12:08.606782 17795 tablet_service.cc:1467] Processing CreateTablet for tablet 8192dccce7e14f3f95b0002061be5123 (DEFAULT_TABLE table=test-table [id=7688b62c169a4571b3329b7155cf5fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:08.606808 18004 tablet_service.cc:1467] Processing CreateTablet for tablet 8192dccce7e14f3f95b0002061be5123 (DEFAULT_TABLE table=test-table [id=7688b62c169a4571b3329b7155cf5fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:08.608091 17795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8192dccce7e14f3f95b0002061be5123. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:08.608256 18004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8192dccce7e14f3f95b0002061be5123. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:08.607847 17595 tablet_service.cc:1467] Processing CreateTablet for tablet 8192dccce7e14f3f95b0002061be5123 (DEFAULT_TABLE table=test-table [id=7688b62c169a4571b3329b7155cf5fd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:08.609095 17595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8192dccce7e14f3f95b0002061be5123. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:08.620546 18138 tablet_bootstrap.cc:492] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: Bootstrap starting.
I20260811 22:12:08.623327 18139 tablet_bootstrap.cc:492] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: Bootstrap starting.
I20260811 22:12:08.625068 18138 tablet_bootstrap.cc:654] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:08.626663 18138 log.cc:824] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:08.627249 18140 tablet_bootstrap.cc:492] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: Bootstrap starting.
I20260811 22:12:08.627508 18139 tablet_bootstrap.cc:654] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:08.628949 18139 log.cc:824] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:08.631053 18140 tablet_bootstrap.cc:654] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:08.631459 18138 tablet_bootstrap.cc:492] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: No bootstrap required, opened a new log
I20260811 22:12:08.631769 18138 ts_tablet_manager.cc:1397] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260811 22:12:08.631951 18139 tablet_bootstrap.cc:492] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: No bootstrap required, opened a new log
I20260811 22:12:08.632237 18139 ts_tablet_manager.cc:1397] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 22:12:08.632295 18140 log.cc:824] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:08.635426 18140 tablet_bootstrap.cc:492] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: No bootstrap required, opened a new log
I20260811 22:12:08.635702 18140 ts_tablet_manager.cc:1397] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 22:12:08.640960 18139 raft_consensus.cc:348] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff079bca43d4114b05767db0a598664" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 43113 } } peers { permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 } } peers { permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 } }
I20260811 22:12:08.641361 18139 raft_consensus.cc:374] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:08.641479 18139 raft_consensus.cc:729] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f0be427b355483b8685ab95db21d210, State: Initialized, Role: FOLLOWER
I20260811 22:12:08.642179 18139 consensus_queue.cc:260] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff079bca43d4114b05767db0a598664" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 43113 } } peers { permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 } } peers { permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 } }
I20260811 22:12:08.642268 18138 raft_consensus.cc:348] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff079bca43d4114b05767db0a598664" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 43113 } } peers { permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 } } peers { permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 } }
I20260811 22:12:08.642747 18138 raft_consensus.cc:374] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:08.642850 18138 raft_consensus.cc:729] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff079bca43d4114b05767db0a598664, State: Initialized, Role: FOLLOWER
I20260811 22:12:08.643627 18106 heartbeater.cc:503] Master 127.10.173.190:41089 was elected leader, sending a full tablet report...
I20260811 22:12:08.643517 18138 consensus_queue.cc:260] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff079bca43d4114b05767db0a598664" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 43113 } } peers { permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 } } peers { permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 } }
I20260811 22:12:08.644364 18139 ts_tablet_manager.cc:1428] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: Time spent starting tablet: real 0.012s user 0.010s sys 0.004s
I20260811 22:12:08.645380 17895 heartbeater.cc:503] Master 127.10.173.190:41089 was elected leader, sending a full tablet report...
I20260811 22:12:08.645946 18138 ts_tablet_manager.cc:1428] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260811 22:12:08.646574 18140 raft_consensus.cc:348] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff079bca43d4114b05767db0a598664" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 43113 } } peers { permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 } } peers { permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 } }
I20260811 22:12:08.647033 18140 raft_consensus.cc:374] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:08.647145 18140 raft_consensus.cc:729] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e7c727860ae41f3af7dcfa17cab599c, State: Initialized, Role: FOLLOWER
I20260811 22:12:08.647778 18140 consensus_queue.cc:260] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff079bca43d4114b05767db0a598664" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 43113 } } peers { permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 } } peers { permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 } }
I20260811 22:12:08.650676 18140 ts_tablet_manager.cc:1428] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
W20260811 22:12:08.693871 17695 tablet.cc:2369] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:12:08.727571 17896 tablet.cc:2369] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:12:08.761479 18107 tablet.cc:2369] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:08.848544 18147 raft_consensus.cc:482] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:08.848945 18147 raft_consensus.cc:504] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff079bca43d4114b05767db0a598664" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 43113 } } peers { permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 } } peers { permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 } }
I20260811 22:12:08.851027 18147 leader_election.cc:288] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f0be427b355483b8685ab95db21d210 (127.10.173.131:36519), 4e7c727860ae41f3af7dcfa17cab599c (127.10.173.129:40293)
I20260811 22:12:08.858070 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.851569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:57815 (local address 127.10.173.131:36519)
0811 22:12:08.851861 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:08.851871 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:08.852637 (+ 766us) server_negotiation.cc:407] Connection header received
0811 22:12:08.852838 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:08.852842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:08.852905 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:08.853009 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:08.853670 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.854489 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.856297 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.857075 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.857273 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.857354 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:08.857675 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 22:12:08.857804 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260811 22:12:08.858119 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.851665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:40833 (local address 127.10.173.129:40293)
0811 22:12:08.851924 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:08.851932 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:08.852736 (+ 804us) server_negotiation.cc:407] Connection header received
0811 22:12:08.852945 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:08.852949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:08.853009 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:08.853108 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:08.853795 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.854595 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.856358 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.857092 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.857273 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.857360 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:08.857732 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 22:12:08.857856 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:12:08.858301 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.851585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40293 (local address 127.10.173.130:40833)
0811 22:12:08.852609 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:08.852637 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:08.852781 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:08.853128 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:08.853132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:08.853156 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:08.853620 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.853629 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.854738 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.854743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.856179 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.856190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.857347 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.857353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.857559 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.857582 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:12:08.857684 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:12:08.857809 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"mutex_wait_us":40,"thread_start_us":512,"threads_started":1}
I20260811 22:12:08.858676 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.851237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:36519 (local address 127.10.173.130:57815)
0811 22:12:08.852503 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:08.852520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:08.852654 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:08.853033 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:08.853037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:08.853063 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:08.853509 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.853520 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.854625 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.854631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.856131 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:08.856147 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.857309 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.857318 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:08.857490 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.857535 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 22:12:08.857643 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:12:08.857780 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1156,"mutex_wait_us":3,"spinlock_wait_cycles":38272,"thread_start_us":938,"threads_started":1}
I20260811 22:12:08.859458 17627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8192dccce7e14f3f95b0002061be5123" candidate_uuid: "6ff079bca43d4114b05767db0a598664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e7c727860ae41f3af7dcfa17cab599c" is_pre_election: true
I20260811 22:12:08.859702 18036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8192dccce7e14f3f95b0002061be5123" candidate_uuid: "6ff079bca43d4114b05767db0a598664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f0be427b355483b8685ab95db21d210" is_pre_election: true
I20260811 22:12:08.860123 17627 raft_consensus.cc:2454] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff079bca43d4114b05767db0a598664 in term 0.
I20260811 22:12:08.860234 18036 raft_consensus.cc:2454] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ff079bca43d4114b05767db0a598664 in term 0.
I20260811 22:12:08.860893 17731 leader_election.cc:302] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4e7c727860ae41f3af7dcfa17cab599c, 6ff079bca43d4114b05767db0a598664, 8f0be427b355483b8685ab95db21d210; no voters:
I20260811 22:12:08.861330 18147 raft_consensus.cc:2781] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:08.861491 18147 raft_consensus.cc:482] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:08.861610 18147 raft_consensus.cc:3037] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:08.863912 18147 raft_consensus.cc:504] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff079bca43d4114b05767db0a598664" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 43113 } } peers { permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 } } peers { permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 } }
I20260811 22:12:08.864575 18147 leader_election.cc:288] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [CANDIDATE]: Term 1 election: Requested vote from peers 8f0be427b355483b8685ab95db21d210 (127.10.173.131:36519), 4e7c727860ae41f3af7dcfa17cab599c (127.10.173.129:40293)
I20260811 22:12:08.865096 18036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8192dccce7e14f3f95b0002061be5123" candidate_uuid: "6ff079bca43d4114b05767db0a598664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f0be427b355483b8685ab95db21d210"
I20260811 22:12:08.865099 17627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8192dccce7e14f3f95b0002061be5123" candidate_uuid: "6ff079bca43d4114b05767db0a598664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e7c727860ae41f3af7dcfa17cab599c"
I20260811 22:12:08.865327 17627 raft_consensus.cc:3037] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:08.865327 18036 raft_consensus.cc:3037] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:08.868687 17627 raft_consensus.cc:2454] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff079bca43d4114b05767db0a598664 in term 1.
I20260811 22:12:08.868686 18036 raft_consensus.cc:2454] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ff079bca43d4114b05767db0a598664 in term 1.
I20260811 22:12:08.869140 17731 leader_election.cc:302] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e7c727860ae41f3af7dcfa17cab599c, 6ff079bca43d4114b05767db0a598664; no voters:
I20260811 22:12:08.869489 18147 raft_consensus.cc:2781] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:08.869959 18147 raft_consensus.cc:686] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 1 LEADER]: Becoming Leader. State: Replica: 6ff079bca43d4114b05767db0a598664, State: Running, Role: LEADER
I20260811 22:12:08.870540 18147 consensus_queue.cc:237] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff079bca43d4114b05767db0a598664" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 43113 } } peers { permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 } } peers { permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 } }
I20260811 22:12:08.875221 17403 catalog_manager.cc:5495] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ff079bca43d4114b05767db0a598664 (127.10.173.130). New cstate: current_term: 1 leader_uuid: "6ff079bca43d4114b05767db0a598664" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ff079bca43d4114b05767db0a598664" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 43113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 } health_report { overall_health: UNKNOWN } } }
I20260811 22:12:08.896001 18128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.890632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51454 (local address 127.10.173.130:43113)
0811 22:12:08.890870 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:08.890874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:08.890893 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:12:08.891093 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:08.891097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:08.891178 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:08.891331 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:08.891653 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.892507 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.893788 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.894243 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.894337 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.894857 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:08.895376 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:08.895505 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:08.895605 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 22:12:08.895674 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:12:08.904374 18036 raft_consensus.cc:1261] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210 [term 1 FOLLOWER]: Refusing update from remote peer 6ff079bca43d4114b05767db0a598664: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:08.904417 17627 raft_consensus.cc:1261] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c [term 1 FOLLOWER]: Refusing update from remote peer 6ff079bca43d4114b05767db0a598664: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:08.905375 18154 consensus_queue.cc:1035] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f0be427b355483b8685ab95db21d210" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:08.905748 18147 consensus_queue.cc:1035] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e7c727860ae41f3af7dcfa17cab599c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:08.933681 17403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:08.933997 17403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58300:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:08.937741 17403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P faa6b7883ce848829ced36ef5d0dd300: Sending DeleteTablet for 3 replicas of tablet 8192dccce7e14f3f95b0002061be5123
I20260811 22:12:08.939112 18004 tablet_service.cc:1514] Processing DeleteTablet for tablet 8192dccce7e14f3f95b0002061be5123 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:08 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:34586
I20260811 22:12:08.939159 17595 tablet_service.cc:1514] Processing DeleteTablet for tablet 8192dccce7e14f3f95b0002061be5123 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:08 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:44288
I20260811 22:12:08.939471 17795 tablet_service.cc:1514] Processing DeleteTablet for tablet 8192dccce7e14f3f95b0002061be5123 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:08 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:51440
I20260811 22:12:08.941821 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 17500
I20260811 22:12:08.951704 18180 tablet_replica.cc:330] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: stopping tablet replica
I20260811 22:12:08.952375 18180 raft_consensus.cc:2229] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:08.953078 18180 raft_consensus.cc:2258] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:08.955757 18180 ts_tablet_manager.cc:1905] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:08.958824 18181 tablet_replica.cc:330] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: stopping tablet replica
I20260811 22:12:08.959394 18181 raft_consensus.cc:2229] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:08.962951 18181 raft_consensus.cc:2258] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:08.966235 18181 ts_tablet_manager.cc:1905] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:08.967437 18180 ts_tablet_manager.cc:1918] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:08.967622 18180 log.cc:1196] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/8192dccce7e14f3f95b0002061be5123
I20260811 22:12:08.968173 18180 ts_tablet_manager.cc:1939] T 8192dccce7e14f3f95b0002061be5123 P 8f0be427b355483b8685ab95db21d210: Deleting consensus metadata
I20260811 22:12:08.968228 18183 tablet_replica.cc:330] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: stopping tablet replica
I20260811 22:12:08.968761 18183 raft_consensus.cc:2229] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:08.969590 18183 raft_consensus.cc:2258] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:08.970183 17388 catalog_manager.cc:4841] TS 8f0be427b355483b8685ab95db21d210 (127.10.173.131:36519): tablet 8192dccce7e14f3f95b0002061be5123 (table test-table [id=7688b62c169a4571b3329b7155cf5fd0]) successfully deleted
I20260811 22:12:08.972097 18183 ts_tablet_manager.cc:1905] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:08.973877 18181 ts_tablet_manager.cc:1918] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:08.974050 18181 log.cc:1196] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/8192dccce7e14f3f95b0002061be5123
Aug 11 22:12:08 dist-test-slave-68xk krb5kdc[17344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:12:08.974673 18181 ts_tablet_manager.cc:1939] T 8192dccce7e14f3f95b0002061be5123 P 4e7c727860ae41f3af7dcfa17cab599c: Deleting consensus metadata
I20260811 22:12:08.976545 17389 catalog_manager.cc:4841] TS 4e7c727860ae41f3af7dcfa17cab599c (127.10.173.129:40293): tablet 8192dccce7e14f3f95b0002061be5123 (table test-table [id=7688b62c169a4571b3329b7155cf5fd0]) successfully deleted
I20260811 22:12:08.980252 18183 ts_tablet_manager.cc:1918] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:08.980475 18183 log.cc:1196] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/8192dccce7e14f3f95b0002061be5123
I20260811 22:12:08.981148 18183 ts_tablet_manager.cc:1939] T 8192dccce7e14f3f95b0002061be5123 P 6ff079bca43d4114b05767db0a598664: Deleting consensus metadata
I20260811 22:12:08.983188 17389 catalog_manager.cc:4841] TS 6ff079bca43d4114b05767db0a598664 (127.10.173.130:43113): tablet 8192dccce7e14f3f95b0002061be5123 (table test-table [id=7688b62c169a4571b3329b7155cf5fd0]) successfully deleted
I20260811 22:12:08.983669 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.943667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.10.173.129:40293)
0811 22:12:08.943924 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:08.943933 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:08.948732 (+ 4799us) server_negotiation.cc:407] Connection header received
0811 22:12:08.948923 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:08.948932 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:08.949062 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:08.949199 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:08.958874 (+ 9675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.960483 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.963080 (+ 2597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.963988 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.964125 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.975903 (+ 11778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:08.975919 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:08.975938 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:08.975992 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:08.979985 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:08.980932 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:08.980941 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:08.980945 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:08.981102 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:08.981578 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:08.981583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:08.981587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:08.982071 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:08.982381 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:08.983013 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 22:12:08.983254 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 22:12:08.985273 17688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:09.138875 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 17500
I20260811 22:12:09.168896 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 17703
Aug 11 22:12:09 dist-test-slave-68xk krb5kdc[17344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:12:09.186141 18128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:09.169807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.10.173.130:43113)
0811 22:12:09.170051 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:09.170057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:09.170078 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:12:09.170189 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:09.170195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:09.170281 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:09.170396 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:09.171707 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:09.172532 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:09.173796 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:09.174256 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:09.174404 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:09.179013 (+ 4609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:09.179033 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:09.179056 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:09.179106 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:09.183181 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:09.183882 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:09.183892 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:09.183896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:09.184078 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:09.184457 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:09.184462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:09.184477 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:09.184930 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:09.185318 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:09.185471 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 22:12:09.185713 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260811 22:12:09.187008 17888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:09.301939 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 17703
I20260811 22:12:09.331413 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 17904
Aug 11 22:12:09 dist-test-slave-68xk krb5kdc[17344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:12:09.346747 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:09.332316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.10.173.131:36519)
0811 22:12:09.332514 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:09.332518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:09.332537 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:12:09.332697 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:09.332703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:09.332768 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:09.332854 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:09.334131 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:09.334916 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:09.336154 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:09.336545 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:09.336641 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:09.341312 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:09.341327 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:09.341343 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:09.341385 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:09.344130 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:09.344756 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:09.344763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:09.344766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:09.344900 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:09.345234 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:09.345237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:09.345239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:09.345583 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:09.345794 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:09.346154 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 22:12:09.346451 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:12:09.347442 18100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:09.437973 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 17904
I20260811 22:12:09.462680 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 17366
I20260811 22:12:09.463987 17458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:09.575367 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 17366
2026-08-11T22:12:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4757 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T22:12:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:09Z Disabled control of system clock
I20260811 22:12:09.672997 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:37947
--webserver_interface=127.10.173.190
--webserver_port=0
--builtin_ntp_servers=127.10.173.148:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:37947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:12:10.011495 18196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:10.011831 18196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:10.012009 18196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:10.019409 18196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:10.019531 18196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:10.019626 18196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:10.019739 18196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:10.019817 18196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:10.029188 18196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:37947
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:37947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:10.030676 18196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:10.032547 18196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:10.044667 18202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:10.044628 18204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:10.044708 18206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:10.046489 18196 server_base.cc:1034] running on GCE node
I20260811 22:12:10.047266 18196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:10.048671 18196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:10.049925 18196 hybrid_clock.cc:648] HybridClock initialized: now 1786486330049887 us; error 53 us; skew 500 ppm
I20260811 22:12:10.050297 18196 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:12:10.052608 18196 webserver.cc:460] Webserver started at http://127.10.173.190:35849/ using document root <none> and password file <none>
I20260811 22:12:10.053318 18196 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:10.053459 18196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:10.053807 18196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:10.056774 18196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "1cd2229c8a1943709aa49cf5fe879dc6"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-68xk"
server_key: "74f85d3c7975d1fa6be11680db0f649d"
server_key_iv: "013c5de2575bbf82cc54850b2245f6d2"
server_key_version: "encryptionkey@0"
I20260811 22:12:10.057571 18196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "1cd2229c8a1943709aa49cf5fe879dc6"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-68xk"
server_key: "74f85d3c7975d1fa6be11680db0f649d"
server_key_iv: "013c5de2575bbf82cc54850b2245f6d2"
server_key_version: "encryptionkey@0"
I20260811 22:12:10.062651 18196 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260811 22:12:10.065912 18219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:10.067188 18196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:10.067374 18196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "1cd2229c8a1943709aa49cf5fe879dc6"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-68xk"
server_key: "74f85d3c7975d1fa6be11680db0f649d"
server_key_iv: "013c5de2575bbf82cc54850b2245f6d2"
server_key_version: "encryptionkey@0"
I20260811 22:12:10.067592 18196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:10.073349 18196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:10.074111 18196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:10.074455 18196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:10.097083 18196 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:37947
I20260811 22:12:10.097105 18294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:37947 every 8 connection(s)
I20260811 22:12:10.098526 18196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:10.102249 18295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:10.105316 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 18196
I20260811 22:12:10.105510 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:10.106194 10934 external_mini_cluster.cc:1428] Setting key 5ed27716535ffbd041cb3caaf1254eb7
I20260811 22:12:10.114660 18299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:10.108312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.10.173.190:37947)
0811 22:12:10.109301 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:10.109312 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:10.109352 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:12:10.110996 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:10.111019 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:10.111454 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:10.111871 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:10.112993 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:10.112999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:10.113010 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:10.113023 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:10.113075 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:10.113089 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:10.113327 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:10.113467 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 22:12:10.113720 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":328,"threads_started":1}
I20260811 22:12:10.114780 18295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6: Bootstrap starting.
I20260811 22:12:10.118170 18295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:10.119251 18295 log.cc:824] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:10.121948 18295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6: No bootstrap required, opened a new log
I20260811 22:12:10.129093 18295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd2229c8a1943709aa49cf5fe879dc6" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37947 } }
I20260811 22:12:10.129499 18295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:10.129618 18295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cd2229c8a1943709aa49cf5fe879dc6, State: Initialized, Role: FOLLOWER
I20260811 22:12:10.130283 18295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd2229c8a1943709aa49cf5fe879dc6" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37947 } }
I20260811 22:12:10.130584 18295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:10.130743 18295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:10.130919 18295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:10.133028 18295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd2229c8a1943709aa49cf5fe879dc6" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37947 } }
I20260811 22:12:10.133530 18295 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cd2229c8a1943709aa49cf5fe879dc6; no voters:
I20260811 22:12:10.134322 18295 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:10.134498 18301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:10.135254 18301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [term 1 LEADER]: Becoming Leader. State: Replica: 1cd2229c8a1943709aa49cf5fe879dc6, State: Running, Role: LEADER
I20260811 22:12:10.135852 18301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd2229c8a1943709aa49cf5fe879dc6" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37947 } }
I20260811 22:12:10.136600 18295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:10.138818 18302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cd2229c8a1943709aa49cf5fe879dc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd2229c8a1943709aa49cf5fe879dc6" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37947 } } }
I20260811 22:12:10.138891 18303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cd2229c8a1943709aa49cf5fe879dc6. Latest consensus state: current_term: 1 leader_uuid: "1cd2229c8a1943709aa49cf5fe879dc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd2229c8a1943709aa49cf5fe879dc6" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37947 } } }
I20260811 22:12:10.139209 18302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:10.139336 18303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:10.141765 18309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:10.150084 18309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:10.159258 18309 catalog_manager.cc:1347] Generated new cluster ID: a3ba358b69c94ba7a7c37a221d4c0f1f
I20260811 22:12:10.159410 18309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:10.183355 18309 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:10.183840 18309 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:10.193529 18309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6: Generated new TSK 0
I20260811 22:12:10.194347 18309 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:12:10.200184 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37947
--builtin_ntp_servers=127.10.173.148:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:12:10.540048 18327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:10.540316 18327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:10.540475 18327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:10.547814 18327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:10.547916 18327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:10.548087 18327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:12:10.557184 18327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:10.558537 18327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:10.560328 18327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:10.573858 18336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:10.573905 18337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:10.573825 18340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:10.579993 18327 server_base.cc:1034] running on GCE node
I20260811 22:12:10.580622 18327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:10.581672 18327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:10.582891 18327 hybrid_clock.cc:648] HybridClock initialized: now 1786486330582851 us; error 52 us; skew 500 ppm
I20260811 22:12:10.583220 18327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:12:10.585628 18327 webserver.cc:460] Webserver started at http://127.10.173.129:33133/ using document root <none> and password file <none>
I20260811 22:12:10.586378 18327 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:10.586542 18327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:10.586887 18327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:10.589883 18327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "66d17e81836a4adda7dfa10508e3f4d1"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-68xk"
server_key: "b9248bde1293bf1957947f43688bd2d4"
server_key_iv: "2bd8fa001eb9da91c409a84dca11d432"
server_key_version: "encryptionkey@0"
I20260811 22:12:10.590780 18327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "66d17e81836a4adda7dfa10508e3f4d1"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-68xk"
server_key: "b9248bde1293bf1957947f43688bd2d4"
server_key_iv: "2bd8fa001eb9da91c409a84dca11d432"
server_key_version: "encryptionkey@0"
I20260811 22:12:10.596205 18327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:12:10.599644 18348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:10.600981 18327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:10.601176 18327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "66d17e81836a4adda7dfa10508e3f4d1"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-68xk"
server_key: "b9248bde1293bf1957947f43688bd2d4"
server_key_iv: "2bd8fa001eb9da91c409a84dca11d432"
server_key_version: "encryptionkey@0"
I20260811 22:12:10.601410 18327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:10.607513 18327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:10.608314 18327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:10.608603 18327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:10.609560 18327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:10.610987 18327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:10.611129 18327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:10.611259 18327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:10.611346 18327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:10.645583 18327 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:37339
I20260811 22:12:10.645640 18503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:37339 every 8 connection(s)
I20260811 22:12:10.646991 18327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:10.653330 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 18327
I20260811 22:12:10.653550 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:12:10.654467 10934 external_mini_cluster.cc:1428] Setting key 930ea1f438b995337dbe556942a1f8fe
I20260811 22:12:10.655629 18506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:10.649644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:44561 (local address 127.10.173.190:37947)
0811 22:12:10.650096 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:10.650101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:10.651258 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 22:12:10.653018 (+ 1760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:10.653024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:10.653082 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:10.653190 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:10.654831 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:10.654835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:10.654839 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:10.654856 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:10.654902 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:10.654915 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:10.655014 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:10.655285 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 22:12:10.655356 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":241,"threads_started":1}
I20260811 22:12:10.656767 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:10.650102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37947 (local address 127.10.173.129:44561)
0811 22:12:10.651086 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:10.651122 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:10.652688 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:10.653514 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:10.653528 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:10.653654 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:10.654441 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:12:10.654447 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:10.654503 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:12:10.654629 (+ 126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:10.655022 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:10.655033 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:10.655041 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:12:10.655305 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 22:12:10.655561 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":327,"threads_started":1}
I20260811 22:12:10.657675 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37947
--builtin_ntp_servers=127.10.173.148:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:12:10.658397 18504 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37947
I20260811 22:12:10.658758 18504 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:10.659466 18504 heartbeater.cc:511] Master 127.10.173.190:37947 requested a full tablet report, sending...
I20260811 22:12:10.661736 18245 ts_manager.cc:194] Registered new tserver with Master: 66d17e81836a4adda7dfa10508e3f4d1 (127.10.173.129:37339)
W20260811 22:12:10.996973 18511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:10.997277 18511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:10.997419 18511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:11.004686 18511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:11.004841 18511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:11.005070 18511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:12:11.015618 18511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:11.016999 18511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:11.018911 18511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:11.033035 18516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:11.033107 18520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:11.033202 18518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:11.033900 18511 server_base.cc:1034] running on GCE node
I20260811 22:12:11.034631 18511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:11.035607 18511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:11.036883 18511 hybrid_clock.cc:648] HybridClock initialized: now 1786486331036814 us; error 51 us; skew 500 ppm
I20260811 22:12:11.037214 18511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:12:11.039615 18511 webserver.cc:460] Webserver started at http://127.10.173.130:41955/ using document root <none> and password file <none>
I20260811 22:12:11.040306 18511 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:11.040443 18511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:11.040779 18511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:11.043810 18511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "867b9360cfdf4299806282bd6403a270"
format_stamp: "Formatted at 2026-08-11 22:12:11 on dist-test-slave-68xk"
server_key: "fd4c9bdc6270faefbe74d671b052b125"
server_key_iv: "fcf691799106dd11c6a445747ca5b1c2"
server_key_version: "encryptionkey@0"
I20260811 22:12:11.044613 18511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "867b9360cfdf4299806282bd6403a270"
format_stamp: "Formatted at 2026-08-11 22:12:11 on dist-test-slave-68xk"
server_key: "fd4c9bdc6270faefbe74d671b052b125"
server_key_iv: "fcf691799106dd11c6a445747ca5b1c2"
server_key_version: "encryptionkey@0"
I20260811 22:12:11.049892 18511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:12:11.053534 18526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.055028 18511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:11.055219 18511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "867b9360cfdf4299806282bd6403a270"
format_stamp: "Formatted at 2026-08-11 22:12:11 on dist-test-slave-68xk"
server_key: "fd4c9bdc6270faefbe74d671b052b125"
server_key_iv: "fcf691799106dd11c6a445747ca5b1c2"
server_key_version: "encryptionkey@0"
I20260811 22:12:11.055445 18511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:11.061087 18511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:11.061863 18511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:11.062137 18511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:11.063125 18511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:11.064491 18511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:11.064630 18511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.064761 18511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:11.064853 18511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.102430 18511 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:34725
I20260811 22:12:11.102483 18687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:34725 every 8 connection(s)
I20260811 22:12:11.103823 18511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:11.112514 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 18511
I20260811 22:12:11.112685 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:12:11.112723 18506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.106827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:40207 (local address 127.10.173.190:37947)
0811 22:12:11.107027 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.107031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.108589 (+ 1558us) server_negotiation.cc:407] Connection header received
0811 22:12:11.110295 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.110301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.110372 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.110507 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.111895 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.111901 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.111906 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.111925 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.111966 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.111982 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.112111 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.112364 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.112434 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:12:11.113269 10934 external_mini_cluster.cc:1428] Setting key d766b1f6485ad0c5945efc5b9a789b0f
I20260811 22:12:11.113750 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.107450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37947 (local address 127.10.173.130:40207)
0811 22:12:11.108420 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.108455 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:11.110022 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:11.110734 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:11.110742 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:11.110829 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:11.111593 (+ 764us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:12:11.111596 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:11.111638 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:12:11.111720 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:11.112141 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:11.112153 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:11.112163 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:12:11.112444 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 22:12:11.112688 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":316,"threads_started":1}
I20260811 22:12:11.115433 18689 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37947
I20260811 22:12:11.115756 18689 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:11.116459 18689 heartbeater.cc:511] Master 127.10.173.190:37947 requested a full tablet report, sending...
I20260811 22:12:11.116438 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37947
--builtin_ntp_servers=127.10.173.148:38115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:12:11.117995 18245 ts_manager.cc:194] Registered new tserver with Master: 867b9360cfdf4299806282bd6403a270 (127.10.173.130:34725)
W20260811 22:12:11.458642 18694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:11.459010 18694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:11.459188 18694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:11.466640 18694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:11.466773 18694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:11.466964 18694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:12:11.477491 18694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:11.478947 18694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:11.480803 18694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:11.494596 18701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:11.494575 18700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:11.495056 18703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:11.495507 18694 server_base.cc:1034] running on GCE node
I20260811 22:12:11.496085 18694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:11.497128 18694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:11.498389 18694 hybrid_clock.cc:648] HybridClock initialized: now 1786486331498322 us; error 53 us; skew 500 ppm
I20260811 22:12:11.498854 18694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:12:11.501432 18694 webserver.cc:460] Webserver started at http://127.10.173.131:39041/ using document root <none> and password file <none>
I20260811 22:12:11.502197 18694 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:11.502297 18694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:11.502712 18694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:11.505849 18694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "19114516eeb0445d885bf486f0f9276b"
format_stamp: "Formatted at 2026-08-11 22:12:11 on dist-test-slave-68xk"
server_key: "625f1bdaa624e765c16e720f1f12fb2f"
server_key_iv: "cbecbb00cc290b8d57145cb2b2c7fae7"
server_key_version: "encryptionkey@0"
I20260811 22:12:11.506816 18694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "19114516eeb0445d885bf486f0f9276b"
format_stamp: "Formatted at 2026-08-11 22:12:11 on dist-test-slave-68xk"
server_key: "625f1bdaa624e765c16e720f1f12fb2f"
server_key_iv: "cbecbb00cc290b8d57145cb2b2c7fae7"
server_key_version: "encryptionkey@0"
I20260811 22:12:11.512151 18694 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 22:12:11.516072 18712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.517519 18694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:12:11.517740 18694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "19114516eeb0445d885bf486f0f9276b"
format_stamp: "Formatted at 2026-08-11 22:12:11 on dist-test-slave-68xk"
server_key: "625f1bdaa624e765c16e720f1f12fb2f"
server_key_iv: "cbecbb00cc290b8d57145cb2b2c7fae7"
server_key_version: "encryptionkey@0"
I20260811 22:12:11.517989 18694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:11.524266 18694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:11.525120 18694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:11.525434 18694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:11.526464 18694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:11.527966 18694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:11.528106 18694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.528256 18694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:11.528362 18694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.566972 18694 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:32841
I20260811 22:12:11.567030 18874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:32841 every 8 connection(s)
I20260811 22:12:11.568430 18694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:11.571194 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 18694
I20260811 22:12:11.571419 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:11.572096 10934 external_mini_cluster.cc:1428] Setting key 487531f08c0ecd4feb4458253538d105
I20260811 22:12:11.578125 18506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.571678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:51153 (local address 127.10.173.190:37947)
0811 22:12:11.571864 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.571868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.573883 (+ 2015us) server_negotiation.cc:407] Connection header received
0811 22:12:11.575818 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.575823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.575879 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.575968 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.577501 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.577505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.577509 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.577524 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.577554 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.577565 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.577656 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.577860 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.577910 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:12:11.579236 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.572480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37947 (local address 127.10.173.131:51153)
0811 22:12:11.573684 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.573744 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:11.575543 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:11.576270 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:11.576279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:11.576384 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:11.577201 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:12:11.577204 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:11.577246 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:12:11.577339 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:11.577662 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:11.577671 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:11.577677 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:12:11.577911 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 22:12:11.578164 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":377,"threads_started":1}
I20260811 22:12:11.580631 18876 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37947
I20260811 22:12:11.580904 18876 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:11.581440 18876 heartbeater.cc:511] Master 127.10.173.190:37947 requested a full tablet report, sending...
I20260811 22:12:11.582837 18245 ts_manager.cc:194] Registered new tserver with Master: 19114516eeb0445d885bf486f0f9276b (127.10.173.131:32841)
I20260811 22:12:11.588228 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:12:11.596065 18506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.593148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59084 (local address 127.10.173.190:37947)
0811 22:12:11.593482 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.593492 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.593987 (+ 495us) server_negotiation.cc:407] Connection header received
0811 22:12:11.594158 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.594162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.594217 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.594305 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.595451 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.595455 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.595458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.595475 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.595507 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.595518 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.595621 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.595699 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.595744 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260811 22:12:11.600379 18245 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59084:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:11.608887 18245 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:11.638540 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.622911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55250 (local address 127.10.173.129:37339)
0811 22:12:11.633252 (+ 10341us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.633263 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.634643 (+ 1380us) server_negotiation.cc:407] Connection header received
0811 22:12:11.634733 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.634741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.634991 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.635174 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.636951 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.636975 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.636996 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.637023 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.637092 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.637121 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.637667 (+ 546us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.637975 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.638087 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10168,"thread_start_us":11486,"threads_started":1}
I20260811 22:12:11.638636 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.623574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:32841 (local address 127.0.0.1:33772)
0811 22:12:11.634210 (+ 10636us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.634245 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:11.634345 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:11.636205 (+ 1860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:11.636209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:11.636229 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:11.637061 (+ 832us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:12:11.637070 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:11.637088 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:12:11.637134 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:11.637792 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:11.637797 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:11.637802 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:12:11.637966 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:12:11.638090 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10495,"thread_start_us":9840,"threads_started":1}
I20260811 22:12:11.639065 18903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.622345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:37339 (local address 127.0.0.1:55250)
0811 22:12:11.633718 (+ 11373us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.633744 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:11.633858 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:11.635213 (+ 1355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:11.635224 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:11.635317 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:11.636266 (+ 949us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:12:11.636270 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:11.636301 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:12:11.636403 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:11.637724 (+ 1321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:11.637737 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:11.637747 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:12:11.638552 (+ 805us) client_negotiation.cc:240] Negotiation successful
0811 22:12:11.638617 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11225,"thread_start_us":10503,"threads_started":1}
I20260811 22:12:11.639333 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.624288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33772 (local address 127.10.173.131:32841)
0811 22:12:11.634543 (+ 10255us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.634551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.634582 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:12:11.634648 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.634654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.634835 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.635026 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.637336 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.637349 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.637366 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.637392 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.637454 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.637483 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.637678 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.638859 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.638954 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10001,"thread_start_us":9862,"threads_started":1}
I20260811 22:12:11.640314 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.622436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.10.173.130:34725)
0811 22:12:11.632783 (+ 10347us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.632796 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.636171 (+ 3375us) server_negotiation.cc:407] Connection header received
0811 22:12:11.636301 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.636310 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.636522 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.636734 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.639285 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.639295 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.639313 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.639346 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.639411 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.639438 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.639586 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.639866 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.639966 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10146,"thread_start_us":9988,"threads_started":1}
I20260811 22:12:11.640461 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.622213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:34725 (local address 127.0.0.1:34872)
0811 22:12:11.633314 (+ 11101us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.633379 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:11.633604 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:11.636725 (+ 3121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:11.636730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:11.636754 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:11.637949 (+ 1195us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:12:11.637952 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:11.637969 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:12:11.638011 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:11.639670 (+ 1659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:11.639676 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:11.639682 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:12:11.639838 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 22:12:11.639889 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10797,"thread_start_us":11645,"threads_started":1}
I20260811 22:12:11.643509 18783 tablet_service.cc:1467] Processing CreateTablet for tablet 3d77adf1c4ff446087b62b40e64450b3 (DEFAULT_TABLE table=test-table [id=b91f99f392ff42ac9022d7a58e894a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:11.643612 18409 tablet_service.cc:1467] Processing CreateTablet for tablet 3d77adf1c4ff446087b62b40e64450b3 (DEFAULT_TABLE table=test-table [id=b91f99f392ff42ac9022d7a58e894a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:11.644356 18593 tablet_service.cc:1467] Processing CreateTablet for tablet 3d77adf1c4ff446087b62b40e64450b3 (DEFAULT_TABLE table=test-table [id=b91f99f392ff42ac9022d7a58e894a40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:11.645076 18783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d77adf1c4ff446087b62b40e64450b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:11.645190 18409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d77adf1c4ff446087b62b40e64450b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:11.645716 18593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d77adf1c4ff446087b62b40e64450b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:11.660677 18907 tablet_bootstrap.cc:492] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: Bootstrap starting.
I20260811 22:12:11.662958 18909 tablet_bootstrap.cc:492] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: Bootstrap starting.
I20260811 22:12:11.665023 18504 heartbeater.cc:503] Master 127.10.173.190:37947 was elected leader, sending a full tablet report...
I20260811 22:12:11.666402 18907 tablet_bootstrap.cc:654] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:11.667153 18909 tablet_bootstrap.cc:654] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:11.668016 18907 log.cc:824] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:11.668471 18909 log.cc:824] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:11.668956 18910 tablet_bootstrap.cc:492] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1: Bootstrap starting.
I20260811 22:12:11.672925 18910 tablet_bootstrap.cc:654] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:11.673981 18910 log.cc:824] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:11.677407 18910 tablet_bootstrap.cc:492] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1: No bootstrap required, opened a new log
I20260811 22:12:11.677729 18910 ts_tablet_manager.cc:1397] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 22:12:11.677814 18907 tablet_bootstrap.cc:492] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: No bootstrap required, opened a new log
I20260811 22:12:11.677810 18909 tablet_bootstrap.cc:492] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: No bootstrap required, opened a new log
I20260811 22:12:11.678128 18909 ts_tablet_manager.cc:1397] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.003s
I20260811 22:12:11.678139 18907 ts_tablet_manager.cc:1397] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20260811 22:12:11.686388 18910 raft_consensus.cc:348] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 } } peers { permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 } } peers { permanent_uuid: "19114516eeb0445d885bf486f0f9276b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 32841 } }
I20260811 22:12:11.686940 18910 raft_consensus.cc:374] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:11.687105 18910 raft_consensus.cc:729] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d17e81836a4adda7dfa10508e3f4d1, State: Initialized, Role: FOLLOWER
I20260811 22:12:11.687959 18910 consensus_queue.cc:260] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 } } peers { permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 } } peers { permanent_uuid: "19114516eeb0445d885bf486f0f9276b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 32841 } }
I20260811 22:12:11.690485 18910 ts_tablet_manager.cc:1428] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1: Time spent starting tablet: real 0.013s user 0.005s sys 0.009s
I20260811 22:12:11.690606 18909 raft_consensus.cc:348] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 } } peers { permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 } } peers { permanent_uuid: "19114516eeb0445d885bf486f0f9276b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 32841 } }
I20260811 22:12:11.690600 18907 raft_consensus.cc:348] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 } } peers { permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 } } peers { permanent_uuid: "19114516eeb0445d885bf486f0f9276b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 32841 } }
I20260811 22:12:11.691157 18909 raft_consensus.cc:374] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:11.691164 18907 raft_consensus.cc:374] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:11.691296 18909 raft_consensus.cc:729] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 867b9360cfdf4299806282bd6403a270, State: Initialized, Role: FOLLOWER
I20260811 22:12:11.691286 18907 raft_consensus.cc:729] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19114516eeb0445d885bf486f0f9276b, State: Initialized, Role: FOLLOWER
I20260811 22:12:11.692301 18909 consensus_queue.cc:260] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 } } peers { permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 } } peers { permanent_uuid: "19114516eeb0445d885bf486f0f9276b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 32841 } }
I20260811 22:12:11.692338 18907 consensus_queue.cc:260] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 } } peers { permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 } } peers { permanent_uuid: "19114516eeb0445d885bf486f0f9276b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 32841 } }
I20260811 22:12:11.694223 18876 heartbeater.cc:503] Master 127.10.173.190:37947 was elected leader, sending a full tablet report...
I20260811 22:12:11.694624 18909 ts_tablet_manager.cc:1428] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: Time spent starting tablet: real 0.016s user 0.014s sys 0.004s
I20260811 22:12:11.694860 18907 ts_tablet_manager.cc:1428] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: Time spent starting tablet: real 0.017s user 0.012s sys 0.005s
I20260811 22:12:11.695363 18689 heartbeater.cc:503] Master 127.10.173.190:37947 was elected leader, sending a full tablet report...
I20260811 22:12:11.697240 18916 raft_consensus.cc:482] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:11.697561 18916 raft_consensus.cc:504] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 } } peers { permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 } } peers { permanent_uuid: "19114516eeb0445d885bf486f0f9276b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 32841 } }
I20260811 22:12:11.699743 18916 leader_election.cc:288] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 867b9360cfdf4299806282bd6403a270 (127.10.173.130:34725), 66d17e81836a4adda7dfa10508e3f4d1 (127.10.173.129:37339)
I20260811 22:12:11.702611 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.699863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:34725 (local address 127.10.173.131:49947)
0811 22:12:11.700266 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.700298 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:11.700402 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:11.700984 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:11.700989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:11.701016 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:11.701760 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:12:11.701764 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:11.701801 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:12:11.701853 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:11.702168 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:11.702173 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:11.702177 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:12:11.702265 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:12:11.702310 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259}
I20260811 22:12:11.702662 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.700356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:49947 (local address 127.10.173.130:34725)
0811 22:12:11.700534 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.700540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.700559 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:12:11.700617 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.700622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.700680 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.700801 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.701986 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.701991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.701995 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.702010 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.702044 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.702057 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.702198 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.702271 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.702335 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:12:11.704021 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.700975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:50445 (local address 127.10.173.129:37339)
0811 22:12:11.701213 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.701223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.702043 (+ 820us) server_negotiation.cc:407] Connection header received
0811 22:12:11.702219 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.702226 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.702292 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.702396 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.703396 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.703402 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.703411 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.703428 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.703458 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.703468 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.703593 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.703674 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.703735 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:12:11.703881 18625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d77adf1c4ff446087b62b40e64450b3" candidate_uuid: "19114516eeb0445d885bf486f0f9276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867b9360cfdf4299806282bd6403a270" is_pre_election: true
I20260811 22:12:11.704201 18918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.700218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:37339 (local address 127.10.173.131:50445)
0811 22:12:11.701956 (+ 1738us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.701974 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:11.702070 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:11.702473 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:11.702478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:11.702500 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:11.703183 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:12:11.703186 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:11.703204 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:12:11.703244 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:11.703567 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:11.703571 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:11.703576 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:12:11.703689 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:12:11.703735 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1594,"thread_start_us":1335,"threads_started":1}
I20260811 22:12:11.704392 18625 raft_consensus.cc:2454] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19114516eeb0445d885bf486f0f9276b in term 0.
I20260811 22:12:11.705374 18443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d77adf1c4ff446087b62b40e64450b3" candidate_uuid: "19114516eeb0445d885bf486f0f9276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66d17e81836a4adda7dfa10508e3f4d1" is_pre_election: true
I20260811 22:12:11.705966 18443 raft_consensus.cc:2454] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19114516eeb0445d885bf486f0f9276b in term 0.
I20260811 22:12:11.706701 18716 leader_election.cc:302] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19114516eeb0445d885bf486f0f9276b, 66d17e81836a4adda7dfa10508e3f4d1; no voters:
I20260811 22:12:11.707127 18916 raft_consensus.cc:2781] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:11.707293 18916 raft_consensus.cc:482] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:11.707407 18916 raft_consensus.cc:3037] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:11.709595 18916 raft_consensus.cc:504] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 } } peers { permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 } } peers { permanent_uuid: "19114516eeb0445d885bf486f0f9276b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 32841 } }
I20260811 22:12:11.710183 18916 leader_election.cc:288] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [CANDIDATE]: Term 1 election: Requested vote from peers 867b9360cfdf4299806282bd6403a270 (127.10.173.130:34725), 66d17e81836a4adda7dfa10508e3f4d1 (127.10.173.129:37339)
I20260811 22:12:11.710636 18625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d77adf1c4ff446087b62b40e64450b3" candidate_uuid: "19114516eeb0445d885bf486f0f9276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "867b9360cfdf4299806282bd6403a270"
I20260811 22:12:11.710763 18443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d77adf1c4ff446087b62b40e64450b3" candidate_uuid: "19114516eeb0445d885bf486f0f9276b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66d17e81836a4adda7dfa10508e3f4d1"
I20260811 22:12:11.710871 18625 raft_consensus.cc:3037] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:11.711000 18443 raft_consensus.cc:3037] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:11.713171 18625 raft_consensus.cc:2454] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19114516eeb0445d885bf486f0f9276b in term 1.
I20260811 22:12:11.713181 18443 raft_consensus.cc:2454] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19114516eeb0445d885bf486f0f9276b in term 1.
I20260811 22:12:11.713608 18715 leader_election.cc:302] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19114516eeb0445d885bf486f0f9276b, 867b9360cfdf4299806282bd6403a270; no voters:
I20260811 22:12:11.713928 18916 raft_consensus.cc:2781] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:11.714557 18916 raft_consensus.cc:686] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 1 LEADER]: Becoming Leader. State: Replica: 19114516eeb0445d885bf486f0f9276b, State: Running, Role: LEADER
I20260811 22:12:11.715054 18916 consensus_queue.cc:237] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 } } peers { permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 } } peers { permanent_uuid: "19114516eeb0445d885bf486f0f9276b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 32841 } }
I20260811 22:12:11.719485 18245 catalog_manager.cc:5495] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b reported cstate change: term changed from 0 to 1, leader changed from <none> to 19114516eeb0445d885bf486f0f9276b (127.10.173.131). New cstate: current_term: 1 leader_uuid: "19114516eeb0445d885bf486f0f9276b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19114516eeb0445d885bf486f0f9276b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 32841 } health_report { overall_health: HEALTHY } } }
I20260811 22:12:11.751006 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.748769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33780 (local address 127.10.173.131:32841)
0811 22:12:11.748991 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.748998 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.749128 (+ 130us) server_negotiation.cc:407] Connection header received
0811 22:12:11.749319 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.749323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.749406 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.749490 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.750386 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.750391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.750394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.750425 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.750459 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.750471 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.750578 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.750660 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.750707 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:12:11.760120 18443 raft_consensus.cc:1261] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1 [term 1 FOLLOWER]: Refusing update from remote peer 19114516eeb0445d885bf486f0f9276b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:11.760150 18625 raft_consensus.cc:1261] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270 [term 1 FOLLOWER]: Refusing update from remote peer 19114516eeb0445d885bf486f0f9276b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:11.761139 18920 consensus_queue.cc:1035] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [LEADER]: Connected to new peer: Peer: permanent_uuid: "66d17e81836a4adda7dfa10508e3f4d1" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:11.761502 18916 consensus_queue.cc:1035] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [LEADER]: Connected to new peer: Peer: permanent_uuid: "867b9360cfdf4299806282bd6403a270" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:11.774274 18927 mvcc.cc:205] Tried to move back new op lower bound from 7317448014878281728 to 7317448014708617216. Current Snapshot: MvccSnapshot[applied={T|T < 7317448014878281728}]
I20260811 22:12:11.790076 18240 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:11.790489 18240 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59084:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:11.794843 18240 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cd2229c8a1943709aa49cf5fe879dc6: Sending DeleteTablet for 3 replicas of tablet 3d77adf1c4ff446087b62b40e64450b3
I20260811 22:12:11.796169 18409 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d77adf1c4ff446087b62b40e64450b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:11 UTC) from {username='slave'} at 127.0.0.1:55250
I20260811 22:12:11.796319 18783 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d77adf1c4ff446087b62b40e64450b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:11 UTC) from {username='slave'} at 127.0.0.1:33772
I20260811 22:12:11.796307 18593 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d77adf1c4ff446087b62b40e64450b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:11 UTC) from {username='slave'} at 127.0.0.1:34872
I20260811 22:12:11.798514 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 18327
I20260811 22:12:11.801654 18944 tablet_replica.cc:330] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: stopping tablet replica
I20260811 22:12:11.802320 18944 raft_consensus.cc:2229] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:11.803135 18944 raft_consensus.cc:2258] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:11.804188 18943 tablet_replica.cc:330] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: stopping tablet replica
I20260811 22:12:11.804792 18943 raft_consensus.cc:2229] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:11.805830 18943 raft_consensus.cc:2258] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:11.806659 18944 ts_tablet_manager.cc:1905] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:11.809118 18943 ts_tablet_manager.cc:1905] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:11.810500 18901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.800074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55258 (local address 127.10.173.129:37339)
0811 22:12:11.800326 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.800336 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.807123 (+ 6787us) server_negotiation.cc:407] Connection header received
0811 22:12:11.807505 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.807514 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.807699 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.807802 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.809388 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.809396 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.809405 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.809433 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.809485 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.809508 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.809652 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.809934 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.810019 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:12:11.811982 18498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:11.817396 18944 ts_tablet_manager.cc:1918] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:11.817585 18944 log.cc:1196] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/3d77adf1c4ff446087b62b40e64450b3
I20260811 22:12:11.818053 18944 ts_tablet_manager.cc:1939] T 3d77adf1c4ff446087b62b40e64450b3 P 867b9360cfdf4299806282bd6403a270: Deleting consensus metadata
I20260811 22:12:11.820129 18222 catalog_manager.cc:4841] TS 867b9360cfdf4299806282bd6403a270 (127.10.173.130:34725): tablet 3d77adf1c4ff446087b62b40e64450b3 (table test-table [id=b91f99f392ff42ac9022d7a58e894a40]) successfully deleted
I20260811 22:12:11.821883 18943 ts_tablet_manager.cc:1918] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:11.822129 18943 log.cc:1196] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/3d77adf1c4ff446087b62b40e64450b3
I20260811 22:12:11.822800 18943 ts_tablet_manager.cc:1939] T 3d77adf1c4ff446087b62b40e64450b3 P 19114516eeb0445d885bf486f0f9276b: Deleting consensus metadata
I20260811 22:12:11.824877 18220 catalog_manager.cc:4841] TS 19114516eeb0445d885bf486f0f9276b (127.10.173.131:32841): tablet 3d77adf1c4ff446087b62b40e64450b3 (table test-table [id=b91f99f392ff42ac9022d7a58e894a40]) successfully deleted
I20260811 22:12:11.907449 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 18327
I20260811 22:12:11.907469 18942 tablet_replica.cc:330] T 3d77adf1c4ff446087b62b40e64450b3 P 66d17e81836a4adda7dfa10508e3f4d1: stopping tablet replica
I20260811 22:12:11.938197 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 18511
W20260811 22:12:11.938225 18223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.173.129:37339 (error 108)
I20260811 22:12:11.941746 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.940966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:37339 (local address 127.0.0.1:55262)
0811 22:12:11.941293 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.941531 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:37339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195}
I20260811 22:12:11.941769 18899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.939057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.10.173.130:34725)
0811 22:12:11.939252 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.939259 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.939375 (+ 116us) server_negotiation.cc:407] Connection header received
0811 22:12:11.939597 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.939605 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.939744 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.939855 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.941056 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.941067 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.941072 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:11.941098 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.941132 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:11.941144 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.941286 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.941380 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.941443 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 22:12:11.942051 18223 catalog_manager.cc:4584] TS 66d17e81836a4adda7dfa10508e3f4d1 (127.10.173.129:37339): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3d77adf1c4ff446087b62b40e64450b3: Network error: Client connection negotiation failed: client connection to 127.10.173.129:37339: connect: Connection refused (error 111)
I20260811 22:12:11.942915 18678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:11.997049 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.996442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:37339 (local address 127.0.0.1:55264)
0811 22:12:11.996716 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.996832 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:37339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 22:12:11.998396 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.997887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:37339 (local address 127.0.0.1:55274)
0811 22:12:11.998074 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.998198 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:37339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:12:12.031858 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 18511
I20260811 22:12:12.059273 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:12.058733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:37339 (local address 127.0.0.1:55278)
0811 22:12:12.058952 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:12.059079 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:37339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 22:12:12.060379 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:12.060028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:37339 (local address 127.0.0.1:55288)
0811 22:12:12.060185 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:12.060267 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:37339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:12:12.065199 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 18694
I20260811 22:12:12.068671 18904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:12.066145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33784 (local address 127.10.173.131:32841)
0811 22:12:12.066330 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:12.066334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:12.066354 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:12:12.066552 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:12.066556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:12.066607 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:12.066698 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:12.067745 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:12.067750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:12.067754 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:12.067769 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:12.067803 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:12.067815 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:12.067932 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:12.068098 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:12:12.068168 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:12:12.069664 18869 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:12.141275 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:12.140799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:37339 (local address 127.0.0.1:55298)
0811 22:12:12.141010 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:12.141117 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:37339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 22:12:12.142562 18897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:12.142074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:37339 (local address 127.0.0.1:55304)
0811 22:12:12.142302 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:12.142373 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:37339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 22:12:12.189348 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 18694
I20260811 22:12:12.219667 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 18196
I20260811 22:12:12.220799 18289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:12.328795 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 18196
2026-08-11T22:12:12Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2749 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 22:12:12.383304 10934 mini_oidc.cc:152] Starting JWKS server
I20260811 22:12:12.386344 10934 webserver.cc:460] Webserver started at https://127.0.0.1:34919/ using document root <none> and password file <none>
I20260811 22:12:12.386817 10934 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:12:12.387528 10934 webserver.cc:460] Webserver started at http://127.0.0.1:33901/ using document root <none> and password file <none>
2026-08-11T22:12:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:12Z Disabled control of system clock
I20260811 22:12:12.420827 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38739
--webserver_interface=127.10.173.190
--webserver_port=0
--builtin_ntp_servers=127.10.173.148:44129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:38739
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34919/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/testchainca.cert with env {}
W20260811 22:12:12.756085 18956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:12.756423 18956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:12.756592 18956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:12.763794 18956 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:12.763947 18956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:12.764045 18956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:12.764168 18956 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:12.764245 18956 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:12:12.764330 18956 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:12:12.764407 18956 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34919/jwks.json
I20260811 22:12:12.774466 18956 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:44129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:38739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34919/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:12.775851 18956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:12.777243 18956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:12.790017 18962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:12.790158 18964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:12.790158 18966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:12.791085 18956 server_base.cc:1034] running on GCE node
I20260811 22:12:12.791999 18956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:12.793491 18956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:12.794734 18956 hybrid_clock.cc:648] HybridClock initialized: now 1786486332794699 us; error 64 us; skew 500 ppm
I20260811 22:12:12.795104 18956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:12:12.797452 18956 webserver.cc:460] Webserver started at http://127.10.173.190:42169/ using document root <none> and password file <none>
I20260811 22:12:12.798169 18956 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:12.798324 18956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:12.798679 18956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:12.801726 18956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "bdd22bd5e4894487b665f78b33a2da93"
format_stamp: "Formatted at 2026-08-11 22:12:12 on dist-test-slave-68xk"
server_key: "6686a2d2a09233b228dc4806e884df79"
server_key_iv: "fea3a6d1902391ec897c708196bb480c"
server_key_version: "encryptionkey@0"
I20260811 22:12:12.802613 18956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "bdd22bd5e4894487b665f78b33a2da93"
format_stamp: "Formatted at 2026-08-11 22:12:12 on dist-test-slave-68xk"
server_key: "6686a2d2a09233b228dc4806e884df79"
server_key_iv: "fea3a6d1902391ec897c708196bb480c"
server_key_version: "encryptionkey@0"
I20260811 22:12:12.807687 18956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260811 22:12:12.811371 18975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:12.812839 18956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:12.813054 18956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "bdd22bd5e4894487b665f78b33a2da93"
format_stamp: "Formatted at 2026-08-11 22:12:12 on dist-test-slave-68xk"
server_key: "6686a2d2a09233b228dc4806e884df79"
server_key_iv: "fea3a6d1902391ec897c708196bb480c"
server_key_version: "encryptionkey@0"
I20260811 22:12:12.813288 18956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:12.831725 18956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:12.832561 18956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:12.832897 18956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:12.853379 18956 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:38739
I20260811 22:12:12.853391 19054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:38739 every 8 connection(s)
I20260811 22:12:12.854959 18956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:12.858788 19055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:12.861716 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 18956
I20260811 22:12:12.861963 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:12.862766 10934 external_mini_cluster.cc:1428] Setting key 4cac88f88ab8199802f6622cc2aef553
I20260811 22:12:12.871534 19055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93: Bootstrap starting.
I20260811 22:12:12.875349 19055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:12.876605 19055 log.cc:824] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:12.877622 19058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:12.864727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.10.173.190:38739)
0811 22:12:12.865810 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:12.865822 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:12.865872 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:12:12.867649 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:12.867673 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:12.868168 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:12.868517 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:12.869187 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:12.870683 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:12.872569 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:12.873336 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:12.873482 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:12.874830 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:12.874838 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:12.874852 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:12.875488 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:12.875629 (+ 141us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:12.875653 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:12.875758 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:12.876044 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 22:12:12.876441 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":430,"threads_started":1}
I20260811 22:12:12.879560 19055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93: No bootstrap required, opened a new log
I20260811 22:12:12.881505 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:12:12.887372 19055 raft_consensus.cc:348] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd22bd5e4894487b665f78b33a2da93" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38739 } }
I20260811 22:12:12.887763 19055 raft_consensus.cc:374] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:12.887876 19055 raft_consensus.cc:729] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdd22bd5e4894487b665f78b33a2da93, State: Initialized, Role: FOLLOWER
I20260811 22:12:12.888480 19055 consensus_queue.cc:260] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd22bd5e4894487b665f78b33a2da93" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38739 } }
I20260811 22:12:12.888731 19055 raft_consensus.cc:388] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:12.888876 19055 raft_consensus.cc:482] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:12.889061 19055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:12.891499 19055 raft_consensus.cc:504] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd22bd5e4894487b665f78b33a2da93" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38739 } }
I20260811 22:12:12.892016 19055 leader_election.cc:302] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bdd22bd5e4894487b665f78b33a2da93; no voters:
I20260811 22:12:12.892585 19055 leader_election.cc:288] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:12.892750 19061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:12.893492 19061 raft_consensus.cc:686] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [term 1 LEADER]: Becoming Leader. State: Replica: bdd22bd5e4894487b665f78b33a2da93, State: Running, Role: LEADER
I20260811 22:12:12.893987 19061 consensus_queue.cc:237] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd22bd5e4894487b665f78b33a2da93" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38739 } }
I20260811 22:12:12.894739 19055 sys_catalog.cc:564] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:12.897017 19064 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bdd22bd5e4894487b665f78b33a2da93. Latest consensus state: current_term: 1 leader_uuid: "bdd22bd5e4894487b665f78b33a2da93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd22bd5e4894487b665f78b33a2da93" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38739 } } }
I20260811 22:12:12.896869 19063 sys_catalog.cc:455] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bdd22bd5e4894487b665f78b33a2da93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bdd22bd5e4894487b665f78b33a2da93" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38739 } } }
I20260811 22:12:12.897369 19064 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:12.897369 19063 sys_catalog.cc:458] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:12.899432 19069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:12.904821 19069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:12.913589 19069 catalog_manager.cc:1347] Generated new cluster ID: ec9cf0038faf48b08d30d0e9677394d7
I20260811 22:12:12.913724 19069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:12.925933 19069 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:12.927305 19069 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:12.935561 19069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bdd22bd5e4894487b665f78b33a2da93: Generated new TSK 0
I20260811 22:12:12.936504 19069 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:12:12.977293 19094 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 22:12:12.977552 19094 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:12:12.979753 19058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:12.956662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58446 (local address 127.10.173.190:38739)
0811 22:12:12.956917 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:12.956926 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:12.957348 (+ 422us) server_negotiation.cc:407] Connection header received
0811 22:12:12.957746 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:12.957750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:12.957849 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:12.957980 (+ 131us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:12:12.958725 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:12.959491 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:12.961606 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:12.962064 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:12.962167 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:12.962861 (+ 694us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:12:12.979201 (+ 16340us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:12:12.979519 (+ 318us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58446: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 22:12:12.979924 19058 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58446: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 22:12:12.981980 19092 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:12.956487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38739 (local address 127.0.0.1:58446)
0811 22:12:12.957209 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:12.957234 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:12.957446 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:12.958049 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:12.958055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:12.958070 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:12:12.958540 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:12.958556 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:12.959645 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:12.959651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:12.961448 (+ 1797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:12.961465 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:12.962236 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:12.962243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:12.962491 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:12.962575 (+ 84us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:12:12.979533 (+ 16958us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:12:12.979922 (+ 389us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.173.190:38739: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":349,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.173.190:38739: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34919/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 22:12:12.984354 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 18956
I20260811 22:12:12.985631 19047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:13.110227 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 18956
2026-08-11T22:12:13Z chronyd exiting
I20260811 22:12:13.158991 10934 test_util.cc:175] -----------------------------------------------
I20260811 22:12:13.159142 10934 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486285256683-10934-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (780 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 22:12:13.163496 10934 mini_oidc.cc:152] Starting JWKS server
I20260811 22:12:13.165419 10934 webserver.cc:460] Webserver started at https://127.0.0.1:43995/ using document root <none> and password file <none>
I20260811 22:12:13.165670 10934 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:12:13.166354 10934 webserver.cc:460] Webserver started at http://127.0.0.1:44957/ using document root <none> and password file <none>
2026-08-11T22:12:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:13Z Disabled control of system clock
I20260811 22:12:13.199678 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:45997
--webserver_interface=127.10.173.190
--webserver_port=0
--builtin_ntp_servers=127.10.173.148:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:45997
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43995/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/testchainca.cert with env {}
W20260811 22:12:13.535262 19104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:13.535559 19104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:13.535718 19104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:13.543287 19104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:13.543398 19104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:13.543473 19104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:13.543573 19104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:13.543651 19104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:12:13.543771 19104 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:12:13.543848 19104 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43995/jwks.json
I20260811 22:12:13.553988 19104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:45997
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:45997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43995/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:13.555397 19104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:13.556732 19104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:13.569361 19112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:13.569320 19109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:13.569409 19110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:13.575659 19104 server_base.cc:1034] running on GCE node
I20260811 22:12:13.576457 19104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:13.577874 19104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:13.579063 19104 hybrid_clock.cc:648] HybridClock initialized: now 1786486333579012 us; error 34 us; skew 500 ppm
I20260811 22:12:13.579459 19104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:12:13.581841 19104 webserver.cc:460] Webserver started at http://127.10.173.190:34715/ using document root <none> and password file <none>
I20260811 22:12:13.582647 19104 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:13.582756 19104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:13.583081 19104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:13.586133 19104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "1e529fc5b4a24479a1030d07f00bb393"
format_stamp: "Formatted at 2026-08-11 22:12:13 on dist-test-slave-68xk"
server_key: "765bb7261772c58e03afb866a43c00b7"
server_key_iv: "9aa6a7cd8d41347392fb86da43335ec7"
server_key_version: "encryptionkey@0"
I20260811 22:12:13.587064 19104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "1e529fc5b4a24479a1030d07f00bb393"
format_stamp: "Formatted at 2026-08-11 22:12:13 on dist-test-slave-68xk"
server_key: "765bb7261772c58e03afb866a43c00b7"
server_key_iv: "9aa6a7cd8d41347392fb86da43335ec7"
server_key_version: "encryptionkey@0"
I20260811 22:12:13.592675 19104 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:12:13.596341 19124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:13.597692 19104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:12:13.597880 19104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "1e529fc5b4a24479a1030d07f00bb393"
format_stamp: "Formatted at 2026-08-11 22:12:13 on dist-test-slave-68xk"
server_key: "765bb7261772c58e03afb866a43c00b7"
server_key_iv: "9aa6a7cd8d41347392fb86da43335ec7"
server_key_version: "encryptionkey@0"
I20260811 22:12:13.598083 19104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:13.624328 19104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:13.625247 19104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:13.625591 19104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:13.644738 19104 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:45997
I20260811 22:12:13.644757 19197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:45997 every 8 connection(s)
I20260811 22:12:13.646153 19104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:13.649930 19198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:13.650821 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 19104
I20260811 22:12:13.651037 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:13.651712 10934 external_mini_cluster.cc:1428] Setting key 5c719d0c3d58efa42985924c8e162a9d
I20260811 22:12:13.663475 19198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393: Bootstrap starting.
I20260811 22:12:13.665869 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.653640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.10.173.190:45997)
0811 22:12:13.654850 (+ 1210us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.654861 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.654912 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:12:13.656429 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.656452 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.656913 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.657241 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:13.657874 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.659467 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.661401 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.662103 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.662322 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.663555 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:13.663562 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:13.663575 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:13.664154 (+ 579us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:13.664239 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:13.664259 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:13.664388 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:13.664615 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 22:12:13.664930 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":413,"threads_started":1}
I20260811 22:12:13.666937 19198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:13.668032 19198 log.cc:824] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:13.669808 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:12:13.671579 19198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393: No bootstrap required, opened a new log
I20260811 22:12:13.679885 19198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e529fc5b4a24479a1030d07f00bb393" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45997 } }
I20260811 22:12:13.680253 19198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:13.680346 19198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e529fc5b4a24479a1030d07f00bb393, State: Initialized, Role: FOLLOWER
I20260811 22:12:13.680929 19198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e529fc5b4a24479a1030d07f00bb393" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45997 } }
I20260811 22:12:13.681138 19198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:13.681241 19198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:13.681375 19198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:13.683665 19198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e529fc5b4a24479a1030d07f00bb393" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45997 } }
I20260811 22:12:13.684155 19198 leader_election.cc:302] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e529fc5b4a24479a1030d07f00bb393; no voters:
I20260811 22:12:13.684764 19198 leader_election.cc:288] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:13.684955 19207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:13.685750 19207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [term 1 LEADER]: Becoming Leader. State: Replica: 1e529fc5b4a24479a1030d07f00bb393, State: Running, Role: LEADER
I20260811 22:12:13.686303 19207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e529fc5b4a24479a1030d07f00bb393" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45997 } }
I20260811 22:12:13.687206 19198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:13.689435 19209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e529fc5b4a24479a1030d07f00bb393. Latest consensus state: current_term: 1 leader_uuid: "1e529fc5b4a24479a1030d07f00bb393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e529fc5b4a24479a1030d07f00bb393" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45997 } } }
I20260811 22:12:13.689522 19208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e529fc5b4a24479a1030d07f00bb393" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e529fc5b4a24479a1030d07f00bb393" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45997 } } }
I20260811 22:12:13.689867 19209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:13.690017 19208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:13.692279 19215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:13.697535 19215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:13.707777 19215 catalog_manager.cc:1347] Generated new cluster ID: 68a1dc366feb4dee93cf466b359ce7b8
I20260811 22:12:13.707926 19215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:13.720861 19215 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:13.722093 19215 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:13.733970 19215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e529fc5b4a24479a1030d07f00bb393: Generated new TSK 0
I20260811 22:12:13.734958 19215 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:12:13.777033 19242 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 22:12:13.777168 19242 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:12:13.778208 19204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.758077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.10.173.190:45997)
0811 22:12:13.758315 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.758324 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.758820 (+ 496us) server_negotiation.cc:407] Connection header received
0811 22:12:13.759054 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.759058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.759139 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.759272 (+ 133us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:12:13.759804 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.760516 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.762626 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.763245 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.763351 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.763971 (+ 620us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:12:13.777665 (+ 13694us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43995/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:12:13.777955 (+ 290us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35062: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43995/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 22:12:13.778366 19204 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35062: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43995/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 22:12:13.778645 19240 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:13.758041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45997 (local address 127.0.0.1:35062)
0811 22:12:13.758709 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:13.758730 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:13.758876 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:13.759322 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:13.759327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:13.759341 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:12:13.759644 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.759665 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.760664 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.760670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.762462 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.762480 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.763445 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.763451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.763665 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.763713 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:12:13.777897 (+ 14184us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43995/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:12:13.778175 (+ 278us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.173.190:45997: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43995/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":384,"threads_started":1}
I20260811 22:12:13.780051 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 19104
I20260811 22:12:13.781313 19192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:13.876731 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 19104
2026-08-11T22:12:13Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (781 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 22:12:13.944334 10934 mini_oidc.cc:152] Starting JWKS server
I20260811 22:12:13.946223 10934 webserver.cc:460] Webserver started at https://127.0.0.1:38775/ using document root <none> and password file <none>
I20260811 22:12:13.946489 10934 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:12:13.947108 10934 webserver.cc:460] Webserver started at http://127.0.0.1:42207/ using document root <none> and password file <none>
2026-08-11T22:12:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:13Z Disabled control of system clock
I20260811 22:12:13.981053 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:40299
--webserver_interface=127.10.173.190
--webserver_port=0
--builtin_ntp_servers=127.10.173.148:33815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:40299
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38775/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:12:14.311677 19251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:14.311954 19251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:14.312114 19251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:14.319199 19251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:14.319303 19251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:14.319373 19251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:14.319460 19251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:14.319548 19251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:12:14.319657 19251 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:12:14.319736 19251 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38775/jwks.json
I20260811 22:12:14.328816 19251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:40299
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:40299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38775/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:14.330178 19251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:14.332012 19251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:14.344516 19259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:14.344543 19260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:14.344453 19263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:14.345459 19251 server_base.cc:1034] running on GCE node
I20260811 22:12:14.346172 19251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:14.347724 19251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:14.348928 19251 hybrid_clock.cc:648] HybridClock initialized: now 1786486334348874 us; error 49 us; skew 500 ppm
I20260811 22:12:14.349290 19251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:12:14.357143 19251 webserver.cc:460] Webserver started at http://127.10.173.190:39181/ using document root <none> and password file <none>
I20260811 22:12:14.357870 19251 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:14.357988 19251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:14.358351 19251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:14.361255 19251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "ba340fc844884673a63e923b84ca9399"
format_stamp: "Formatted at 2026-08-11 22:12:14 on dist-test-slave-68xk"
server_key: "acaa2c9f5919e365e6ba6f3b5178643e"
server_key_iv: "3665b98c2a8d7581c04bbdfaa460f075"
server_key_version: "encryptionkey@0"
I20260811 22:12:14.362051 19251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "ba340fc844884673a63e923b84ca9399"
format_stamp: "Formatted at 2026-08-11 22:12:14 on dist-test-slave-68xk"
server_key: "acaa2c9f5919e365e6ba6f3b5178643e"
server_key_iv: "3665b98c2a8d7581c04bbdfaa460f075"
server_key_version: "encryptionkey@0"
I20260811 22:12:14.367472 19251 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:12:14.371161 19270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:14.372570 19251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:14.372756 19251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "ba340fc844884673a63e923b84ca9399"
format_stamp: "Formatted at 2026-08-11 22:12:14 on dist-test-slave-68xk"
server_key: "acaa2c9f5919e365e6ba6f3b5178643e"
server_key_iv: "3665b98c2a8d7581c04bbdfaa460f075"
server_key_version: "encryptionkey@0"
I20260811 22:12:14.372992 19251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:14.393364 19251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:14.394213 19251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:14.394608 19251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:14.415726 19251 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:40299
I20260811 22:12:14.415764 19347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:40299 every 8 connection(s)
I20260811 22:12:14.417258 19251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:14.421159 19349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:14.422211 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 19251
I20260811 22:12:14.422518 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:14.423264 10934 external_mini_cluster.cc:1428] Setting key 868006b57333c94fcc9045117b524e14
I20260811 22:12:14.433424 19349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399: Bootstrap starting.
I20260811 22:12:14.437234 19349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:14.438294 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:14.425364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.10.173.190:40299)
0811 22:12:14.426669 (+ 1305us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:14.426681 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:14.426730 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:12:14.428519 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:14.428543 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:14.429077 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:14.429447 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:14.430067 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:14.431648 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:14.433570 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:14.434351 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:14.434623 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:14.435880 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:14.435888 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:14.435901 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:14.436475 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:14.436573 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:14.436595 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:14.436682 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:14.436947 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 22:12:14.437369 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":486,"threads_started":1}
I20260811 22:12:14.438326 19349 log.cc:824] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:14.441819 19349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399: No bootstrap required, opened a new log
I20260811 22:12:14.442970 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:12:14.450398 19349 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba340fc844884673a63e923b84ca9399" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40299 } }
I20260811 22:12:14.450997 19349 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:14.451179 19349 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba340fc844884673a63e923b84ca9399, State: Initialized, Role: FOLLOWER
I20260811 22:12:14.451890 19349 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba340fc844884673a63e923b84ca9399" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40299 } }
I20260811 22:12:14.452178 19349 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:14.452342 19349 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:14.452538 19349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:14.454982 19349 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba340fc844884673a63e923b84ca9399" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40299 } }
I20260811 22:12:14.455535 19349 leader_election.cc:302] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba340fc844884673a63e923b84ca9399; no voters:
I20260811 22:12:14.456270 19349 leader_election.cc:288] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:14.456419 19355 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:14.457216 19355 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [term 1 LEADER]: Becoming Leader. State: Replica: ba340fc844884673a63e923b84ca9399, State: Running, Role: LEADER
I20260811 22:12:14.457748 19355 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba340fc844884673a63e923b84ca9399" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40299 } }
I20260811 22:12:14.458583 19349 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:14.460892 19357 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba340fc844884673a63e923b84ca9399. Latest consensus state: current_term: 1 leader_uuid: "ba340fc844884673a63e923b84ca9399" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba340fc844884673a63e923b84ca9399" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40299 } } }
I20260811 22:12:14.460821 19356 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba340fc844884673a63e923b84ca9399" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba340fc844884673a63e923b84ca9399" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40299 } } }
I20260811 22:12:14.461292 19357 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:14.461292 19356 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:14.464527 19374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:14.469870 19374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:14.478463 19374 catalog_manager.cc:1347] Generated new cluster ID: 23b12648e27a44478919fb90c7e95db5
I20260811 22:12:14.478583 19374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:14.496364 19374 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:14.498307 19374 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:14.522821 19374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba340fc844884673a63e923b84ca9399: Generated new TSK 0
I20260811 22:12:14.523948 19374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:12:14.555171 19352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:14.545326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44488 (local address 127.10.173.190:40299)
0811 22:12:14.545584 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:14.545591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:14.545974 (+ 383us) server_negotiation.cc:407] Connection header received
0811 22:12:14.546255 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:14.546259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:14.546337 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:14.546551 (+ 214us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:12:14.547178 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:14.548017 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:14.550198 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:14.550981 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:14.551096 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:14.551760 (+ 664us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:12:14.554312 (+ 2552us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38775/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:12:14.554846 (+ 534us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44488: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38775/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 22:12:14.555310 19352 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44488: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38775/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 22:12:14.557265 19398 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:14.545215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:40299 (local address 127.0.0.1:44488)
0811 22:12:14.545858 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:14.545879 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:14.546039 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:14.546566 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:14.546573 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:14.546593 (+ 20us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:12:14.546935 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:14.546948 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:14.548251 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:14.548257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:14.549970 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:14.549986 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:14.551199 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:14.551206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:14.551418 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:14.551467 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:12:14.554577 (+ 3110us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38775/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:12:14.556778 (+ 2201us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.173.190:40299: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38775/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":1831,"thread_start_us":342,"threads_started":1}
I20260811 22:12:14.558328 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 19251
I20260811 22:12:14.559667 19343 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 22:12:14.567397 19399 webserver.cc:533] Webserver: sslize failed:
E20260811 22:12:14.567488 19399 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:12:14.654201 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 19251
2026-08-11T22:12:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (775 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T22:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:14Z Disabled control of system clock
I20260811 22:12:14.753424 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:46593
--webserver_interface=127.10.173.190
--webserver_port=0
--builtin_ntp_servers=127.10.173.148:35823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:46593
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/testchainca.cert with env {}
W20260811 22:12:15.092952 19406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:15.093554 19406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:15.093724 19406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:15.105325 19406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:15.105437 19406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:15.105527 19406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:15.105608 19406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:15.105690 19406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:12:15.105780 19406 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:12:15.105854 19406 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 22:12:15.116133 19406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:46593
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:46593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:15.117522 19406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:15.118913 19406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:15.131615 19413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:15.131678 19418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:15.131647 19414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:15.132599 19406 server_base.cc:1034] running on GCE node
I20260811 22:12:15.133471 19406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:15.135095 19406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:15.136377 19406 hybrid_clock.cc:648] HybridClock initialized: now 1786486335136372 us; error 80 us; skew 500 ppm
I20260811 22:12:15.136812 19406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:12:15.139640 19406 webserver.cc:460] Webserver started at http://127.10.173.190:43485/ using document root <none> and password file <none>
I20260811 22:12:15.140413 19406 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:15.140537 19406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:15.140882 19406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:15.144459 19406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "e911dfedc4db4d6886723bf004c1a889"
format_stamp: "Formatted at 2026-08-11 22:12:15 on dist-test-slave-68xk"
server_key: "6d864dd299aa4bde18fed9bb6e5b44c0"
server_key_iv: "293f1280c908ed611ccb30fcfee06394"
server_key_version: "encryptionkey@0"
I20260811 22:12:15.145531 19406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "e911dfedc4db4d6886723bf004c1a889"
format_stamp: "Formatted at 2026-08-11 22:12:15 on dist-test-slave-68xk"
server_key: "6d864dd299aa4bde18fed9bb6e5b44c0"
server_key_iv: "293f1280c908ed611ccb30fcfee06394"
server_key_version: "encryptionkey@0"
I20260811 22:12:15.150844 19406 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:12:15.154328 19426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:15.156136 19406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:15.156359 19406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "e911dfedc4db4d6886723bf004c1a889"
format_stamp: "Formatted at 2026-08-11 22:12:15 on dist-test-slave-68xk"
server_key: "6d864dd299aa4bde18fed9bb6e5b44c0"
server_key_iv: "293f1280c908ed611ccb30fcfee06394"
server_key_version: "encryptionkey@0"
I20260811 22:12:15.156629 19406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:15.187438 19406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:15.188417 19406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:15.188759 19406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:15.208613 19406 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:46593
I20260811 22:12:15.208633 19512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:46593 every 8 connection(s)
I20260811 22:12:15.209975 19406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:15.213989 19513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:15.217097 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 19406
I20260811 22:12:15.217324 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:15.218158 10934 external_mini_cluster.cc:1428] Setting key 47ac67f8b38061f432d4f39144716eea
I20260811 22:12:15.226495 19513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889: Bootstrap starting.
I20260811 22:12:15.229766 19513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:15.230993 19513 log.cc:824] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:15.232225 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:15.220377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.10.173.190:46593)
0811 22:12:15.221370 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:15.221382 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:15.221424 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:12:15.223124 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:15.223146 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:15.223616 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:15.224019 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:15.224617 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:15.225889 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:15.227334 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:15.228028 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:15.228236 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:15.229518 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:15.229525 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:15.229540 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:12:15.230143 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:15.230250 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:12:15.230272 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:15.230402 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:15.230678 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 22:12:15.230979 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":305,"threads_started":1}
I20260811 22:12:15.233995 19513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889: No bootstrap required, opened a new log
I20260811 22:12:15.235983 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:12:15.241786 19513 raft_consensus.cc:348] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e911dfedc4db4d6886723bf004c1a889" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46593 } }
I20260811 22:12:15.242156 19513 raft_consensus.cc:374] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:15.242287 19513 raft_consensus.cc:729] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e911dfedc4db4d6886723bf004c1a889, State: Initialized, Role: FOLLOWER
I20260811 22:12:15.242894 19513 consensus_queue.cc:260] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e911dfedc4db4d6886723bf004c1a889" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46593 } }
I20260811 22:12:15.243137 19513 raft_consensus.cc:388] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:15.243294 19513 raft_consensus.cc:482] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:15.243471 19513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:15.245564 19513 raft_consensus.cc:504] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e911dfedc4db4d6886723bf004c1a889" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46593 } }
I20260811 22:12:15.246071 19513 leader_election.cc:302] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e911dfedc4db4d6886723bf004c1a889; no voters:
I20260811 22:12:15.246735 19513 leader_election.cc:288] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:15.246876 19520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:15.248085 19520 raft_consensus.cc:686] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [term 1 LEADER]: Becoming Leader. State: Replica: e911dfedc4db4d6886723bf004c1a889, State: Running, Role: LEADER
I20260811 22:12:15.248713 19520 consensus_queue.cc:237] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e911dfedc4db4d6886723bf004c1a889" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46593 } }
I20260811 22:12:15.249015 19513 sys_catalog.cc:564] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:15.251755 19522 sys_catalog.cc:455] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e911dfedc4db4d6886723bf004c1a889. Latest consensus state: current_term: 1 leader_uuid: "e911dfedc4db4d6886723bf004c1a889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e911dfedc4db4d6886723bf004c1a889" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46593 } } }
I20260811 22:12:15.252115 19522 sys_catalog.cc:458] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:15.251865 19521 sys_catalog.cc:455] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e911dfedc4db4d6886723bf004c1a889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e911dfedc4db4d6886723bf004c1a889" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46593 } } }
I20260811 22:12:15.252295 19521 sys_catalog.cc:458] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:15.254271 19529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:15.259829 19529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:15.269043 19529 catalog_manager.cc:1347] Generated new cluster ID: c114dca6a68d49638e227fa9fd5684e7
I20260811 22:12:15.269177 19529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:15.301137 19529 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:15.303043 19529 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:15.312345 19529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e911dfedc4db4d6886723bf004c1a889: Generated new TSK 0
I20260811 22:12:15.313424 19529 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:12:15.432710 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:15.339104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.10.173.190:46593)
0811 22:12:15.339304 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:15.339308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:15.339427 (+ 119us) server_negotiation.cc:407] Connection header received
0811 22:12:15.339662 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:15.339666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:15.339750 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:15.339883 (+ 133us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:12:15.340435 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:15.341184 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:15.343358 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:15.343812 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:15.343929 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:15.344507 (+ 578us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:12:15.432060 (+ 87553us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:12:15.432447 (+ 387us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48312: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 22:12:15.432906 19517 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48312: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 22:12:15.433140 19557 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:15.338677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46593 (local address 127.0.0.1:48312)
0811 22:12:15.339314 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:15.339335 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:15.339492 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:15.339930 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:15.339936 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:15.339949 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:12:15.340267 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:15.340281 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:15.341345 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:15.341350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:15.343205 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:15.343221 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:15.343988 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:15.343996 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:15.344193 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:15.344241 (+ 48us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:12:15.432395 (+ 88154us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:12:15.432730 (+ 335us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.173.190:46593: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":298,"threads_started":1}
I20260811 22:12:15.434285 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 19406
I20260811 22:12:15.435640 19503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:15.542564 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 19406
2026-08-11T22:12:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (864 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (442 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (429 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:16 dist-test-slave-68xk krb5kdc[19585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:16 dist-test-slave-68xk krb5kdc[19585](info): set up 2 sockets
Aug 11 22:12:16 dist-test-slave-68xk krb5kdc[19585](info): commencing operation
krb5kdc: starting...
W20260811 22:12:17.781986 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:12:17 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:12:18.046766 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:37237
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:42923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:37237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:18.384410 19607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:18.384763 19607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:18.384940 19607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:18.392081 19607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:18.392241 19607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:18.392359 19607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:18.392474 19607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:18.392576 19607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:18.402606 19607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:42923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:37237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:37237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:18.404050 19607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:18.405383 19607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:18.417999 19616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:18.417961 19618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:18.418038 19620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:18.418928 19607 server_base.cc:1034] running on GCE node
I20260811 22:12:18.419854 19607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:18.421309 19607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:18.422536 19607 hybrid_clock.cc:648] HybridClock initialized: now 1786486338422464 us; error 58 us; skew 500 ppm
I20260811 22:12:18.422921 19607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:18 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486338, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:18.426307 19607 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:12:18.428043 19607 webserver.cc:460] Webserver started at http://127.10.173.190:43535/ using document root <none> and password file <none>
I20260811 22:12:18.428800 19607 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:18.428942 19607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:18.429309 19607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:18.432541 19607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "46569c5e66c44527a3128f37c28d5a65"
format_stamp: "Formatted at 2026-08-11 22:12:18 on dist-test-slave-68xk"
server_key: "df6e6585bf4151655ce04172bc9b1d70"
server_key_iv: "4e0c981c8b77cc1cea38d8000bbfb054"
server_key_version: "encryptionkey@0"
I20260811 22:12:18.433359 19607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "46569c5e66c44527a3128f37c28d5a65"
format_stamp: "Formatted at 2026-08-11 22:12:18 on dist-test-slave-68xk"
server_key: "df6e6585bf4151655ce04172bc9b1d70"
server_key_iv: "4e0c981c8b77cc1cea38d8000bbfb054"
server_key_version: "encryptionkey@0"
I20260811 22:12:18.438804 19607 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:12:18.442574 19631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:18.444116 19607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:18.444344 19607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "46569c5e66c44527a3128f37c28d5a65"
format_stamp: "Formatted at 2026-08-11 22:12:18 on dist-test-slave-68xk"
server_key: "df6e6585bf4151655ce04172bc9b1d70"
server_key_iv: "4e0c981c8b77cc1cea38d8000bbfb054"
server_key_version: "encryptionkey@0"
I20260811 22:12:18.444582 19607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:18.468371 19607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:18.473003 19607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:18.473366 19607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:18.494226 19607 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:37237
I20260811 22:12:18.494274 19703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:37237 every 8 connection(s)
I20260811 22:12:18.495766 19607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:18.499704 19705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:18.501667 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 19607
I20260811 22:12:18.501912 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:18.502851 10934 external_mini_cluster.cc:1428] Setting key f5444faf956b7b4f76ca6b5896b1375a
I20260811 22:12:18.513172 19705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65: Bootstrap starting.
I20260811 22:12:18.517103 19705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:18.518455 19705 log.cc:824] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:12:18 dist-test-slave-68xk krb5kdc[19585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:18.521194 19705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65: No bootstrap required, opened a new log
I20260811 22:12:18.527963 19711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:18.505183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41708 (local address 127.10.173.190:37237)
0811 22:12:18.506102 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:18.506115 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:18.506163 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:12:18.508095 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:18.508119 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:18.508516 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:18.508844 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:18.510692 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:18.512202 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:18.513923 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:18.514738 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:18.514881 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:18.519945 (+ 5064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:18.519953 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:18.519968 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:18.520010 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:18.523544 (+ 3534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:18.524334 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:18.524348 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:18.524354 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:18.524533 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:18.524955 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:18.524971 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:18.524975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:18.525576 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:18.525914 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:18.526459 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 22:12:18.526912 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":442,"threads_started":1}
I20260811 22:12:18.529196 19705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46569c5e66c44527a3128f37c28d5a65" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37237 } }
I20260811 22:12:18.529577 19705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:18.529700 19705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46569c5e66c44527a3128f37c28d5a65, State: Initialized, Role: FOLLOWER
I20260811 22:12:18.530284 19705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46569c5e66c44527a3128f37c28d5a65" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37237 } }
I20260811 22:12:18.530628 19705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:18.530786 19705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:18.530973 19705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:18.533191 19705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46569c5e66c44527a3128f37c28d5a65" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37237 } }
I20260811 22:12:18.533684 19705 leader_election.cc:302] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46569c5e66c44527a3128f37c28d5a65; no voters:
I20260811 22:12:18.534236 19705 leader_election.cc:288] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:18.534400 19713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:18.535656 19713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [term 1 LEADER]: Becoming Leader. State: Replica: 46569c5e66c44527a3128f37c28d5a65, State: Running, Role: LEADER
I20260811 22:12:18.536257 19713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46569c5e66c44527a3128f37c28d5a65" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37237 } }
I20260811 22:12:18.536412 19705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:18.539433 19715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46569c5e66c44527a3128f37c28d5a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46569c5e66c44527a3128f37c28d5a65" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37237 } } }
I20260811 22:12:18.539404 19717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46569c5e66c44527a3128f37c28d5a65. Latest consensus state: current_term: 1 leader_uuid: "46569c5e66c44527a3128f37c28d5a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46569c5e66c44527a3128f37c28d5a65" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37237 } } }
I20260811 22:12:18.539846 19715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:18.539846 19717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:18.541000 19726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:18.546737 19726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:18.560240 19726 catalog_manager.cc:1347] Generated new cluster ID: 0df0cd8a77c54c06851a61a5bb11c0db
I20260811 22:12:18.560377 19726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:18.592979 19726 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:18.594358 19726 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:18.609395 19726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46569c5e66c44527a3128f37c28d5a65: Generated new TSK 0
I20260811 22:12:18.610282 19726 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:12:18.691229 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37237
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:42923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:19.052781 19744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:19.053071 19744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:19.053249 19744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:19.060532 19744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:19.060643 19744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:19.060856 19744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:12:19.070551 19744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:42923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:19.071826 19744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:19.073215 19744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:19.086628 19750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:19.086623 19752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:19.086568 19755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:19.087744 19744 server_base.cc:1034] running on GCE node
I20260811 22:12:19.088449 19744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:19.089365 19744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:19.090615 19744 hybrid_clock.cc:648] HybridClock initialized: now 1786486339090560 us; error 66 us; skew 500 ppm
I20260811 22:12:19.090934 19744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:19 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486339, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:19.093986 19744 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:12:19.095448 19744 webserver.cc:460] Webserver started at http://127.10.173.129:33583/ using document root <none> and password file <none>
I20260811 22:12:19.096136 19744 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:19.096289 19744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:19.096623 19744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:19.099586 19744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "6aa878a359a44de48e9ded2de3086f79"
format_stamp: "Formatted at 2026-08-11 22:12:19 on dist-test-slave-68xk"
server_key: "2d035fb27a46ed0d1fc9eaa514c4f1b7"
server_key_iv: "ea3d1c0d3189ce6f1b78d30c97befe2f"
server_key_version: "encryptionkey@0"
I20260811 22:12:19.100390 19744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "6aa878a359a44de48e9ded2de3086f79"
format_stamp: "Formatted at 2026-08-11 22:12:19 on dist-test-slave-68xk"
server_key: "2d035fb27a46ed0d1fc9eaa514c4f1b7"
server_key_iv: "ea3d1c0d3189ce6f1b78d30c97befe2f"
server_key_version: "encryptionkey@0"
I20260811 22:12:19.105576 19744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 22:12:19.109656 19764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:19.111181 19744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:12:19.111374 19744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "6aa878a359a44de48e9ded2de3086f79"
format_stamp: "Formatted at 2026-08-11 22:12:19 on dist-test-slave-68xk"
server_key: "2d035fb27a46ed0d1fc9eaa514c4f1b7"
server_key_iv: "ea3d1c0d3189ce6f1b78d30c97befe2f"
server_key_version: "encryptionkey@0"
I20260811 22:12:19.111646 19744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:19.183977 19744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:19.188964 19744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:19.189255 19744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:19.190284 19744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:19.191798 19744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:19.191882 19744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:19.192073 19744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:19.192138 19744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:19.230738 19744 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:39857
I20260811 22:12:19.230820 19930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:39857 every 8 connection(s)
I20260811 22:12:19.232137 19744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:19.233220 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 19744
I20260811 22:12:19.233481 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:12:19.234328 10934 external_mini_cluster.cc:1428] Setting key 07297598506cc72735e3c08f3eeedb9d
Aug 11 22:12:19 dist-test-slave-68xk krb5kdc[19585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486339, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:19.257694 19933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:19.235408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:43677 (local address 127.10.173.190:37237)
0811 22:12:19.235919 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:19.235926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:19.236742 (+ 816us) server_negotiation.cc:407] Connection header received
0811 22:12:19.238649 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:19.238659 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:19.238762 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:19.238901 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:19.240457 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:19.241735 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:19.243612 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:19.244495 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:19.244662 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:19.249109 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:19.249120 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:19.249129 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:19.249208 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:19.253533 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:19.254612 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:19.254621 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:19.254628 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:19.254816 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:19.255342 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:19.255349 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:19.255355 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:19.255756 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:19.255915 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:19.256782 (+ 867us) server_negotiation.cc:299] Negotiation successful
0811 22:12:19.257112 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":269,"threads_started":1}
I20260811 22:12:19.258369 19936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:19.235665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37237 (local address 127.10.173.129:43677)
0811 22:12:19.236566 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:19.236612 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:19.238305 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:19.239100 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:19.239108 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:19.239666 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:19.240220 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:19.240237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:19.241930 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:19.241935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:19.243445 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:19.243459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:19.244831 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:19.244844 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:19.245150 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:19.245933 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:19.245965 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:19.248822 (+ 2857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:19.253825 (+ 5003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:19.253833 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:19.253848 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:19.254368 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:19.254953 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:19.254957 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:19.254959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:19.255204 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:19.255903 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:19.255909 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:19.256349 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 22:12:19.256621 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 22:12:19.257038 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":374,"threads_started":1}
I20260811 22:12:19.260234 19931 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37237
I20260811 22:12:19.260632 19931 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:19.261626 19931 heartbeater.cc:511] Master 127.10.173.190:37237 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
I20260811 22:12:19.264225 19655 ts_manager.cc:194] Registered new tserver with Master: 6aa878a359a44de48e9ded2de3086f79 (127.10.173.129:39857)
I20260811 22:12:19.265712 19655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:43677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:12:19.296162 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37237
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:42923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:19.635054 19940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:19.635372 19940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:19.635545 19940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:19.642926 19940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:19.643087 19940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:19.643270 19940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:12:19.653293 19940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:42923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:19.654670 19940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:19.655999 19940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:19.670331 19950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:19.670259 19951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:19.670352 19954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:19.671193 19940 server_base.cc:1034] running on GCE node
I20260811 22:12:19.671766 19940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:19.672955 19940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:19.674238 19940 hybrid_clock.cc:648] HybridClock initialized: now 1786486339674159 us; error 56 us; skew 500 ppm
I20260811 22:12:19.674705 19940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:19 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486339, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:19.679522 19940 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:12:19.684531 19940 webserver.cc:460] Webserver started at http://127.10.173.130:43363/ using document root <none> and password file <none>
I20260811 22:12:19.685504 19940 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:19.685657 19940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:19.686023 19940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:19.690747 19940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "713da21005bb425eabba22c957e07d3c"
format_stamp: "Formatted at 2026-08-11 22:12:19 on dist-test-slave-68xk"
server_key: "f3888a3c1293788b2873232202bde2e6"
server_key_iv: "108b27ddb991cc2bf1e4134a17026fcc"
server_key_version: "encryptionkey@0"
I20260811 22:12:19.691885 19940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "713da21005bb425eabba22c957e07d3c"
format_stamp: "Formatted at 2026-08-11 22:12:19 on dist-test-slave-68xk"
server_key: "f3888a3c1293788b2873232202bde2e6"
server_key_iv: "108b27ddb991cc2bf1e4134a17026fcc"
server_key_version: "encryptionkey@0"
I20260811 22:12:19.700142 19940 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260811 22:12:19.707120 19965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:19.709087 19940 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.002s
I20260811 22:12:19.709299 19940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "713da21005bb425eabba22c957e07d3c"
format_stamp: "Formatted at 2026-08-11 22:12:19 on dist-test-slave-68xk"
server_key: "f3888a3c1293788b2873232202bde2e6"
server_key_iv: "108b27ddb991cc2bf1e4134a17026fcc"
server_key_version: "encryptionkey@0"
I20260811 22:12:19.709584 19940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:19.729744 19940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:19.734218 19940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:19.734619 19940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:19.735646 19940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:19.737123 19940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:19.737226 19940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:19.737377 19940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:19.737453 19940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:19.774256 19940 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:40329
I20260811 22:12:19.774307 20130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:40329 every 8 connection(s)
I20260811 22:12:19.775601 19940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:19.783636 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 19940
I20260811 22:12:19.783834 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:12:19.784449 10934 external_mini_cluster.cc:1428] Setting key d9a2a01638b952a1025909082897c8cc
Aug 11 22:12:19 dist-test-slave-68xk krb5kdc[19585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486339, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:19.799363 20133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:19.778360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:52859 (local address 127.10.173.190:37237)
0811 22:12:19.778887 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:19.778893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:19.779945 (+ 1052us) server_negotiation.cc:407] Connection header received
0811 22:12:19.782016 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:19.782022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:19.782125 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:19.782259 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:19.784390 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:19.785293 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:19.787589 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:19.788070 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:19.788183 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:19.793133 (+ 4950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:19.793139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:19.793143 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:19.793179 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:19.796100 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:19.796892 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:19.796897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:19.796899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:19.797033 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:19.797500 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:19.797504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:19.797506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:19.797777 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:19.797950 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:19.798860 (+ 910us) server_negotiation.cc:299] Negotiation successful
0811 22:12:19.799051 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":253,"threads_started":1}
I20260811 22:12:19.800951 20136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:19.778976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37237 (local address 127.10.173.130:52859)
0811 22:12:19.779772 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:19.779817 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:19.781718 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:19.782549 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:19.782558 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:19.783214 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:19.784155 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:19.784192 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:19.785467 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:19.785473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:19.787403 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:19.787419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:19.788331 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:19.788342 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:19.788744 (+ 402us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:19.789618 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:19.789659 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:19.792852 (+ 3193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:19.796261 (+ 3409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:19.796270 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:19.796288 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:19.796746 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:19.797151 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:19.797159 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:19.797163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:19.797364 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:19.797911 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:19.797920 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:19.798390 (+ 470us) client_negotiation.cc:769] Sending connection context
0811 22:12:19.798736 (+ 346us) client_negotiation.cc:240] Negotiation successful
0811 22:12:19.799327 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":316,"threads_started":1}
I20260811 22:12:19.802645 20131 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37237
I20260811 22:12:19.802999 20131 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:19.803759 20131 heartbeater.cc:511] Master 127.10.173.190:37237 requested a full tablet report, sending...
I20260811 22:12:19.805455 19655 ts_manager.cc:194] Registered new tserver with Master: 713da21005bb425eabba22c957e07d3c (127.10.173.130:40329)
I20260811 22:12:19.806828 19655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:52859
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:12:19.846112 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37237
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:42923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:20.189003 20139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:20.189357 20139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:20.189545 20139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:20.196802 20139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:20.196975 20139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:20.197171 20139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:12:20.207247 20139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:42923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37237
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:20.208628 20139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:20.209967 20139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:20.224676 20145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:20.224606 20149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:20.224646 20153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:20.225843 20139 server_base.cc:1034] running on GCE node
I20260811 22:12:20.226554 20139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:20.227560 20139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:20.228842 20139 hybrid_clock.cc:648] HybridClock initialized: now 1786486340228782 us; error 68 us; skew 500 ppm
I20260811 22:12:20.229182 20139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:20.232587 20139 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:12:20.234107 20139 webserver.cc:460] Webserver started at http://127.10.173.131:41029/ using document root <none> and password file <none>
I20260811 22:12:20.234839 20139 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:20.234977 20139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:20.235294 20139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:20.238324 20139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "ea64c15ea0774eb6919a8024799ae2fc"
format_stamp: "Formatted at 2026-08-11 22:12:20 on dist-test-slave-68xk"
server_key: "6b3b38d784f9591aff5b33e4db8ba699"
server_key_iv: "e5a8a6ed69e1e3fc2042f9866eefc397"
server_key_version: "encryptionkey@0"
I20260811 22:12:20.239226 20139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "ea64c15ea0774eb6919a8024799ae2fc"
format_stamp: "Formatted at 2026-08-11 22:12:20 on dist-test-slave-68xk"
server_key: "6b3b38d784f9591aff5b33e4db8ba699"
server_key_iv: "e5a8a6ed69e1e3fc2042f9866eefc397"
server_key_version: "encryptionkey@0"
I20260811 22:12:20.244488 20139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:12:20.248188 20162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:20.249646 20139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:12:20.249879 20139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "ea64c15ea0774eb6919a8024799ae2fc"
format_stamp: "Formatted at 2026-08-11 22:12:20 on dist-test-slave-68xk"
server_key: "6b3b38d784f9591aff5b33e4db8ba699"
server_key_iv: "e5a8a6ed69e1e3fc2042f9866eefc397"
server_key_version: "encryptionkey@0"
I20260811 22:12:20.250065 20139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:20.261533 20139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:20.265615 20139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:20.265869 20139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:20.266865 20139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:20.268683 19931 heartbeater.cc:503] Master 127.10.173.190:37237 was elected leader, sending a full tablet report...
I20260811 22:12:20.269016 20139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:20.269110 20139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:20.269275 20139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:20.269352 20139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:20.305153 20139 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:43131
I20260811 22:12:20.305258 20336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:43131 every 8 connection(s)
I20260811 22:12:20.306581 20139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:20.310700 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 20139
I20260811 22:12:20.310902 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:20.311530 10934 external_mini_cluster.cc:1428] Setting key 411112fdaed37330d57119cef1a18cb3
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:20.328552 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:20.309472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:60443 (local address 127.10.173.190:37237)
0811 22:12:20.310082 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:20.310087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:20.311065 (+ 978us) server_negotiation.cc:407] Connection header received
0811 22:12:20.313315 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:20.313319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:20.313401 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:20.313491 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:20.315664 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.316700 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.318383 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.318913 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.319028 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:20.322371 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:20.322376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:20.322379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:20.322458 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:20.325240 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.326017 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.326024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.326029 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.326232 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.326733 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.326738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.326742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.327136 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:20.327263 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:20.328100 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 22:12:20.328281 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":316,"threads_started":1}
I20260811 22:12:20.329596 20340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:20.309865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37237 (local address 127.10.173.131:60443)
0811 22:12:20.310819 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:20.310880 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:20.313009 (+ 2129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:20.313786 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:20.313794 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:20.314613 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:20.315484 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:20.315505 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.316850 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.316854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:20.318217 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:20.318240 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.319114 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.319119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:20.319315 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:20.319950 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:20.319972 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:20.322174 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:20.325381 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:20.325390 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:20.325405 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:20.325909 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:20.326368 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:20.326372 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:20.326375 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:20.326613 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:20.327271 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:20.327279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:20.327730 (+ 451us) client_negotiation.cc:769] Sending connection context
0811 22:12:20.327967 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 22:12:20.328355 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":348,"threads_started":1}
I20260811 22:12:20.331040 20337 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37237
I20260811 22:12:20.331341 20337 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:20.332041 20337 heartbeater.cc:511] Master 127.10.173.190:37237 requested a full tablet report, sending...
I20260811 22:12:20.333457 19655 ts_manager.cc:194] Registered new tserver with Master: ea64c15ea0774eb6919a8024799ae2fc (127.10.173.131:43131)
I20260811 22:12:20.334302 19655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:60443
I20260811 22:12:20.339838 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/test-admin.keytab.
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:20.372705 10934 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 22:12:20.378525 20347 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:20.375838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38378 (local address 127.10.173.129:39857)
0811 22:12:20.376475 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:20.376482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:20.376512 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:12:20.376585 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:20.376593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:20.376807 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:20.376943 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:20.377967 (+ 1024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38378: BlockingRecv error: recv got EOF from 127.0.0.1:38378 (error 108)
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":313,"threads_started":1}
W20260811 22:12:20.378765 10934 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5cc)
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:20.380347 10934 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:20.381721 10934 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:12:20.397959 20347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:20.383411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38390 (local address 127.10.173.129:39857)
0811 22:12:20.383618 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:20.383626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:20.383939 (+ 313us) server_negotiation.cc:407] Connection header received
0811 22:12:20.384135 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:20.384139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:20.384202 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:20.384306 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:20.385363 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.386368 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.387672 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.388227 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.388371 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:20.392167 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:20.392180 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:20.392195 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:20.392233 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:20.395183 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.395973 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.395981 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.395983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.396112 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.396516 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.396520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.396522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.396882 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:20.397107 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:20.397466 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 22:12:20.397694 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 22:12:20.404448 20347 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:20.402400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38394 (local address 127.10.173.129:39857)
0811 22:12:20.402653 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:20.402661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:20.402979 (+ 318us) server_negotiation.cc:407] Connection header received
0811 22:12:20.403166 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:20.403171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:20.403266 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:20.403370 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:20.404158 (+ 788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38394: BlockingRecv error: recv got EOF from 127.0.0.1:38394 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 22:12:20.404706 10934 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:20.406350 10934 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:20.407852 10934 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:12:20.423589 20347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:20.409364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38410 (local address 127.10.173.129:39857)
0811 22:12:20.409573 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:20.409581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:20.409829 (+ 248us) server_negotiation.cc:407] Connection header received
0811 22:12:20.410012 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:20.410022 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:20.410078 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:20.410180 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:20.411218 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.411918 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.413135 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.413565 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.413711 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:20.418043 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:20.418049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:20.418052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:20.418098 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:20.420905 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.421591 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.421597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.421601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.421739 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.422155 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.422166 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.422169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.422500 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:20.422776 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:20.423083 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 22:12:20.423311 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 22:12:20.428771 20347 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:20.426690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38426 (local address 127.10.173.129:39857)
0811 22:12:20.426995 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:20.427002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:20.427260 (+ 258us) server_negotiation.cc:407] Connection header received
0811 22:12:20.427453 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:20.427459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:20.427543 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:20.427649 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:20.428477 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38426: BlockingRecv error: recv got EOF from 127.0.0.1:38426 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260811 22:12:20.428996 10934 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486285256683-10934-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:20.430840 10934 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:20.432304 10934 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:12:20.447324 20347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:20.433729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.10.173.129:39857)
0811 22:12:20.433947 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:20.433953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:20.434104 (+ 151us) server_negotiation.cc:407] Connection header received
0811 22:12:20.434292 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:20.434299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:20.434366 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:20.434511 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:20.435443 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.436191 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.437687 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.438202 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.438308 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:20.441884 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:20.441889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:20.441893 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:20.441925 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:20.444705 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.445375 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.445379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.445382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.445518 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.445955 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.445958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.445960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.446229 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:20.446524 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:20.446775 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 22:12:20.447002 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:12:20.449098 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 19744
I20260811 22:12:20.462597 20347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:20.449924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.10.173.129:39857)
0811 22:12:20.450182 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:20.450188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:20.450556 (+ 368us) server_negotiation.cc:407] Connection header received
0811 22:12:20.450748 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:20.450760 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:20.450816 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:20.450918 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:20.451907 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.452570 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.453840 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.454253 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.454387 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:20.456519 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:20.456524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:20.456527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:20.456556 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:20.459381 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.460150 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.460158 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.460164 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.460417 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.460927 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.460933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.460937 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.461229 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:20.461545 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:20.461903 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 22:12:20.462196 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 22:12:20.463245 19926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:20.548696 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 19744
I20260811 22:12:20.570245 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 19940
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:12:20.586294 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:20.571235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52814 (local address 127.10.173.130:40329)
0811 22:12:20.571779 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:20.571786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:20.571812 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:12:20.571874 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:20.571880 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:20.572086 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:20.572266 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:20.573301 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.574322 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.575647 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.576202 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.576347 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:20.580291 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:20.580303 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:20.580317 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:20.580349 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:20.583336 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.584026 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.584034 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.584036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.584163 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.584553 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.584559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.584562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.584935 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:20.585292 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:20.585682 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 22:12:20.585969 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":283,"threads_started":1}
I20260811 22:12:20.587776 20125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:20.671850 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 19940
I20260811 22:12:20.693473 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 20139
Aug 11 22:12:20 dist-test-slave-68xk krb5kdc[19585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:12:20.710611 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:20.694633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36578 (local address 127.10.173.131:43131)
0811 22:12:20.695327 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:20.695334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:20.695364 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:12:20.695428 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:20.695434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:20.695657 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:20.695786 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:20.696887 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.697927 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.699296 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:20.699866 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:20.699971 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:20.704185 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:20.704199 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:20.704215 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:20.704252 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:20.707349 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.708160 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.708169 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.708171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.708308 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:20.708768 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:20.708773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:20.708776 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:20.709176 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:20.709545 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:20.709886 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 22:12:20.710203 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":359,"threads_started":1}
I20260811 22:12:20.712013 20330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:20.796101 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 20139
I20260811 22:12:20.823474 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 19607
I20260811 22:12:20.824851 19699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:20.930896 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 19607
2026-08-11T22:12:20Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4525 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:21 dist-test-slave-68xk krb5kdc[20378](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:21 dist-test-slave-68xk krb5kdc[20378](info): set up 2 sockets
Aug 11 22:12:21 dist-test-slave-68xk krb5kdc[20378](info): commencing operation
krb5kdc: starting...
W20260811 22:12:22.280974 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:12:22 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:22Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.190.keytab.
Entry for principal oryx/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.190.keytab.
I20260811 22:12:22.540102 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:32997
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.190.keytab
--principal=oryx/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:32997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:22.879621 20398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:22.879930 20398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:22.880101 20398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:22.887408 20398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:22.887518 20398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:22.887626 20398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:22.887708 20398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:22.887773 20398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:22.897629 20398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:32997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.190.keytab
--principal=oryx/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:32997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:22.899085 20398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:22.900516 20398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:22.912572 20405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:22.912618 20409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:22.912643 20406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:22.913406 20398 server_base.cc:1034] running on GCE node
I20260811 22:12:22.914265 20398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:22.915851 20398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:22.917137 20398 hybrid_clock.cc:648] HybridClock initialized: now 1786486342917077 us; error 73 us; skew 500 ppm
I20260811 22:12:22.917589 20398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:22 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:22.921041 20398 init.cc:377] Logged in from keytab as oryx/127.10.173.190@KRBTEST.COM (short username oryx)
I20260811 22:12:22.922703 20398 webserver.cc:460] Webserver started at http://127.10.173.190:32995/ using document root <none> and password file <none>
I20260811 22:12:22.923478 20398 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:22.923641 20398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:22.924037 20398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:22.927450 20398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "11e46714a91e4356a74769c0ceaaed9c"
format_stamp: "Formatted at 2026-08-11 22:12:22 on dist-test-slave-68xk"
server_key: "d4b78800ba37aad2f364141dd4c7c76f"
server_key_iv: "4cca976a6c2af4547c50c4c283522afc"
server_key_version: "encryptionkey@0"
I20260811 22:12:22.928298 20398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "11e46714a91e4356a74769c0ceaaed9c"
format_stamp: "Formatted at 2026-08-11 22:12:22 on dist-test-slave-68xk"
server_key: "d4b78800ba37aad2f364141dd4c7c76f"
server_key_iv: "4cca976a6c2af4547c50c4c283522afc"
server_key_version: "encryptionkey@0"
I20260811 22:12:22.934001 20398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:12:22.937808 20420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:22.939302 20398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:12:22.939499 20398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "11e46714a91e4356a74769c0ceaaed9c"
format_stamp: "Formatted at 2026-08-11 22:12:22 on dist-test-slave-68xk"
server_key: "d4b78800ba37aad2f364141dd4c7c76f"
server_key_iv: "4cca976a6c2af4547c50c4c283522afc"
server_key_version: "encryptionkey@0"
I20260811 22:12:22.939713 20398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:22.951329 20398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:22.955732 20398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:22.956100 20398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:22.975417 20398 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:32997
I20260811 22:12:22.975454 20497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:32997 every 8 connection(s)
I20260811 22:12:22.976958 20398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:22.980664 20499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:22.985491 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 20398
I20260811 22:12:22.985719 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:22.986641 10934 external_mini_cluster.cc:1428] Setting key fe9da22a901d80f8d94e3e37feeded45
I20260811 22:12:22.992530 20499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c: Bootstrap starting.
I20260811 22:12:22.996263 20499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:22.997545 20499 log.cc:824] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:23.001685 20499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c: No bootstrap required, opened a new log
Aug 11 22:12:23 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:23.013757 20499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } }
I20260811 22:12:23.014286 20499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:23.014398 20499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11e46714a91e4356a74769c0ceaaed9c, State: Initialized, Role: FOLLOWER
I20260811 22:12:23.015134 20499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } }
I20260811 22:12:23.015421 20499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:23.015578 20499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:23.015770 20499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:23.016233 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.988867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36440 (local address 127.10.173.190:32997)
0811 22:12:22.989785 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:22.989797 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:22.989845 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:12:22.991742 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:22.991767 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:22.992145 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:22.992505 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:22.994513 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.995975 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.997871 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.999237 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.999442 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.004819 (+ 5377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:23.004835 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:23.004864 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:23.004943 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:23.010768 (+ 5825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:23.011816 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:23.011826 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:23.011837 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:23.012117 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:23.012604 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:23.012609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:23.012612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:23.013452 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:23.013787 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.014227 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.014768 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":429,"threads_started":1}
I20260811 22:12:23.018857 20499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } }
I20260811 22:12:23.019428 20499 leader_election.cc:302] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11e46714a91e4356a74769c0ceaaed9c; no voters:
I20260811 22:12:23.020058 20499 leader_election.cc:288] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:23.020288 20505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:23.021042 20505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 1 LEADER]: Becoming Leader. State: Replica: 11e46714a91e4356a74769c0ceaaed9c, State: Running, Role: LEADER
I20260811 22:12:23.021683 20505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } }
I20260811 22:12:23.022136 20499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:23.024788 20506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } }
I20260811 22:12:23.024832 20507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11e46714a91e4356a74769c0ceaaed9c. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } }
I20260811 22:12:23.025242 20507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:23.025242 20506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:23.037580 20513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:23.043330 20513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:23.054497 20513 catalog_manager.cc:1347] Generated new cluster ID: 35427e1bb15c4054a01441bf811a0e08
I20260811 22:12:23.054633 20513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:23.062527 20513 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:23.064579 20513 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:23.071480 20513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c: Generated new TSK 0
I20260811 22:12:23.072283 20513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab.
Entry for principal oryx/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab.
I20260811 22:12:23.155233 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32997
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab
--principal=oryx/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:23.523573 20539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:23.523901 20539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:23.524068 20539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:23.531647 20539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:23.531807 20539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:23.532001 20539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:12:23.542129 20539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab
--principal=oryx/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:23.543607 20539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:23.545136 20539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:23.559041 20546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:23.559056 20547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:23.559084 20550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:23.559453 20539 server_base.cc:1034] running on GCE node
I20260811 22:12:23.560268 20539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:23.561208 20539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:23.562510 20539 hybrid_clock.cc:648] HybridClock initialized: now 1786486343562462 us; error 58 us; skew 500 ppm
I20260811 22:12:23.562855 20539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:23 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486343, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:23.566581 20539 init.cc:377] Logged in from keytab as oryx/127.10.173.129@KRBTEST.COM (short username oryx)
I20260811 22:12:23.568012 20539 webserver.cc:460] Webserver started at http://127.10.173.129:39649/ using document root <none> and password file <none>
I20260811 22:12:23.568708 20539 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:23.568846 20539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:23.569166 20539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:23.572273 20539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "f2a43230c04d4b64bd8fd1f05e505914"
format_stamp: "Formatted at 2026-08-11 22:12:23 on dist-test-slave-68xk"
server_key: "8b3367ad69b2c866eb2a7b024a8d4c60"
server_key_iv: "ecdffeb508e9e2ec42303e44d95ec033"
server_key_version: "encryptionkey@0"
I20260811 22:12:23.573060 20539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "f2a43230c04d4b64bd8fd1f05e505914"
format_stamp: "Formatted at 2026-08-11 22:12:23 on dist-test-slave-68xk"
server_key: "8b3367ad69b2c866eb2a7b024a8d4c60"
server_key_iv: "ecdffeb508e9e2ec42303e44d95ec033"
server_key_version: "encryptionkey@0"
I20260811 22:12:23.578105 20539 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 22:12:23.581487 20559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:23.582935 20539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:23.583105 20539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "f2a43230c04d4b64bd8fd1f05e505914"
format_stamp: "Formatted at 2026-08-11 22:12:23 on dist-test-slave-68xk"
server_key: "8b3367ad69b2c866eb2a7b024a8d4c60"
server_key_iv: "ecdffeb508e9e2ec42303e44d95ec033"
server_key_version: "encryptionkey@0"
I20260811 22:12:23.583305 20539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:23.596014 20539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:23.600476 20539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:23.600791 20539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:23.601747 20539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:23.603214 20539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:23.603295 20539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:23.603434 20539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:23.603495 20539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:23.637368 20539 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:45861
I20260811 22:12:23.637408 20738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:45861 every 8 connection(s)
I20260811 22:12:23.638651 20539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:23.644874 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 20539
I20260811 22:12:23.645112 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:12:23.645979 10934 external_mini_cluster.cc:1428] Setting key a1194d874398e24cc100512860a7664a
Aug 11 22:12:23 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486343, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:23.661123 20741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.641178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:49287 (local address 127.10.173.190:32997)
0811 22:12:23.641643 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:23.641648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:23.642663 (+ 1015us) server_negotiation.cc:407] Connection header received
0811 22:12:23.644641 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:23.644645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:23.644771 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:23.644866 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:23.646962 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.648079 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.650586 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.650984 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.651141 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.655426 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:23.655431 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:23.655434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:23.655465 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:23.658030 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:23.658844 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:23.658848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:23.658851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:23.658976 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:23.659423 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:23.659434 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:23.659437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:23.659691 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:23.659866 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.660527 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.660764 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":226,"threads_started":1}
I20260811 22:12:23.662076 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.641770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.129:49287)
0811 22:12:23.642515 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:23.642553 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:23.644328 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:23.645166 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:23.645177 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:23.645935 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:23.646757 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.646779 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.648249 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.648254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.650354 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.650379 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.651260 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.651270 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.651603 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.652399 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:23.652434 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:23.655193 (+ 2759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:23.658175 (+ 2982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:23.658185 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:23.658200 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:23.658709 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:23.659092 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:23.659098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:23.659101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:23.659300 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:23.659824 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:23.659833 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:23.660158 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 22:12:23.660427 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 22:12:23.660944 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":270,"threads_started":1}
I20260811 22:12:23.663712 20739 heartbeater.cc:347] Connected to a master server at 127.10.173.190:32997
I20260811 22:12:23.664060 20739 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:23.664968 20739 heartbeater.cc:511] Master 127.10.173.190:32997 requested a full tablet report, sending...
I20260811 22:12:23.667568 20448 ts_manager.cc:194] Registered new tserver with Master: f2a43230c04d4b64bd8fd1f05e505914 (127.10.173.129:45861)
I20260811 22:12:23.669911 20448 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:49287
WARNING: no policy specified for oryx/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab.
Entry for principal oryx/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab.
I20260811 22:12:23.706388 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32997
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab
--principal=oryx/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:24.046774 20748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:24.047065 20748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:24.047170 20748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:24.054401 20748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:24.054540 20748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:24.054654 20748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:12:24.064242 20748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab
--principal=oryx/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:24.065448 20748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:24.066818 20748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:24.080327 20754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:24.080361 20758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:24.080295 20756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:24.081557 20748 server_base.cc:1034] running on GCE node
I20260811 22:12:24.082258 20748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:24.083317 20748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:24.084534 20748 hybrid_clock.cc:648] HybridClock initialized: now 1786486344084473 us; error 48 us; skew 500 ppm
I20260811 22:12:24.084898 20748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:24 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486344, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:24.088681 20748 init.cc:377] Logged in from keytab as oryx/127.10.173.130@KRBTEST.COM (short username oryx)
I20260811 22:12:24.090392 20748 webserver.cc:460] Webserver started at http://127.10.173.130:44947/ using document root <none> and password file <none>
I20260811 22:12:24.091221 20748 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:24.091390 20748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:24.091747 20748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:24.095088 20748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "c94f10d8f018483fbc787ed2b498e390"
format_stamp: "Formatted at 2026-08-11 22:12:24 on dist-test-slave-68xk"
server_key: "6f7f21e4ef14093824e0f499c2a85f7a"
server_key_iv: "e824b27a01d5828e5e5fdd5f71f5669a"
server_key_version: "encryptionkey@0"
I20260811 22:12:24.095958 20748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "c94f10d8f018483fbc787ed2b498e390"
format_stamp: "Formatted at 2026-08-11 22:12:24 on dist-test-slave-68xk"
server_key: "6f7f21e4ef14093824e0f499c2a85f7a"
server_key_iv: "e824b27a01d5828e5e5fdd5f71f5669a"
server_key_version: "encryptionkey@0"
I20260811 22:12:24.101552 20748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:12:24.105527 20768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.107160 20748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:24.107371 20748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "c94f10d8f018483fbc787ed2b498e390"
format_stamp: "Formatted at 2026-08-11 22:12:24 on dist-test-slave-68xk"
server_key: "6f7f21e4ef14093824e0f499c2a85f7a"
server_key_iv: "e824b27a01d5828e5e5fdd5f71f5669a"
server_key_version: "encryptionkey@0"
I20260811 22:12:24.107623 20748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:24.132210 20748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:24.137041 20748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:24.137355 20748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:24.138366 20748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:24.139775 20748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:24.139854 20748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.140028 20748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:24.140084 20748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.175028 20748 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:36603
I20260811 22:12:24.175127 20942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:36603 every 8 connection(s)
I20260811 22:12:24.176384 20748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:24.181305 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 20748
I20260811 22:12:24.181485 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:12:24.181974 10934 external_mini_cluster.cc:1428] Setting key 45550bcec53e23120ecadeb3e8827550
Aug 11 22:12:24 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486344, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:24.200465 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.179135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:34485 (local address 127.10.173.190:32997)
0811 22:12:24.179661 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:24.179666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:24.180532 (+ 866us) server_negotiation.cc:407] Connection header received
0811 22:12:24.182907 (+ 2375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:24.182913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:24.182998 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:24.183176 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:24.185210 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:24.186552 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:24.188523 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:24.189332 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:24.189494 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:24.193471 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:24.193481 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:24.193490 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:24.193541 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:24.196722 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:24.197800 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:24.197804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:24.197807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:24.197947 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:24.198513 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:24.198517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:24.198519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:24.198816 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:24.198940 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:24.199978 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0811 22:12:24.200182 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":276,"threads_started":1}
I20260811 22:12:24.202162 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.179611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.130:34485)
0811 22:12:24.180358 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.180404 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:24.182519 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:24.183471 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:24.183484 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:24.184333 (+ 849us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:24.184980 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:24.184999 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:24.186741 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:24.186746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:24.188363 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:24.188377 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:24.189563 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:24.189570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:24.189789 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:24.190576 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:24.190601 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:24.193228 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:24.196896 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:24.196909 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:24.196931 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:24.197666 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:24.198072 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:24.198081 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:24.198088 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:24.198362 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:24.198967 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:24.198979 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:24.199512 (+ 533us) client_negotiation.cc:769] Sending connection context
0811 22:12:24.199852 (+ 340us) client_negotiation.cc:240] Negotiation successful
0811 22:12:24.200508 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":293,"threads_started":1}
I20260811 22:12:24.203874 20943 heartbeater.cc:347] Connected to a master server at 127.10.173.190:32997
I20260811 22:12:24.204221 20943 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:24.205016 20943 heartbeater.cc:511] Master 127.10.173.190:32997 requested a full tablet report, sending...
I20260811 22:12:24.206780 20448 ts_manager.cc:194] Registered new tserver with Master: c94f10d8f018483fbc787ed2b498e390 (127.10.173.130:36603)
I20260811 22:12:24.207741 20448 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:34485
WARNING: no policy specified for oryx/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab.
Entry for principal oryx/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab.
I20260811 22:12:24.246459 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32997
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab
--principal=oryx/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:24.583130 20956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:24.583465 20956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:24.583636 20956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:24.591326 20956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:24.591480 20956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:24.591660 20956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:12:24.601619 20956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab
--principal=oryx/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:24.603070 20956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:24.604487 20956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:24.618230 20963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:24.618253 20966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:24.618460 20964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:24.618901 20956 server_base.cc:1034] running on GCE node
I20260811 22:12:24.619577 20956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:24.620515 20956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:24.621786 20956 hybrid_clock.cc:648] HybridClock initialized: now 1786486344621728 us; error 63 us; skew 500 ppm
I20260811 22:12:24.622166 20956 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:24 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486344, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:24.625685 20956 init.cc:377] Logged in from keytab as oryx/127.10.173.131@KRBTEST.COM (short username oryx)
I20260811 22:12:24.627265 20956 webserver.cc:460] Webserver started at http://127.10.173.131:36667/ using document root <none> and password file <none>
I20260811 22:12:24.628001 20956 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:24.628230 20956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:24.628561 20956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:24.631594 20956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "3d7de0600ef3442180518f144d68006b"
format_stamp: "Formatted at 2026-08-11 22:12:24 on dist-test-slave-68xk"
server_key: "6abcf3972b221b756e3fd401b96f7987"
server_key_iv: "3f77641b5f44cc5539ce7ccf70285316"
server_key_version: "encryptionkey@0"
I20260811 22:12:24.632401 20956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "3d7de0600ef3442180518f144d68006b"
format_stamp: "Formatted at 2026-08-11 22:12:24 on dist-test-slave-68xk"
server_key: "6abcf3972b221b756e3fd401b96f7987"
server_key_iv: "3f77641b5f44cc5539ce7ccf70285316"
server_key_version: "encryptionkey@0"
I20260811 22:12:24.637830 20956 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:12:24.641620 20979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.643054 20956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:12:24.643244 20956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "3d7de0600ef3442180518f144d68006b"
format_stamp: "Formatted at 2026-08-11 22:12:24 on dist-test-slave-68xk"
server_key: "6abcf3972b221b756e3fd401b96f7987"
server_key_iv: "3f77641b5f44cc5539ce7ccf70285316"
server_key_version: "encryptionkey@0"
I20260811 22:12:24.643460 20956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:24.673240 20956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:24.673996 20739 heartbeater.cc:503] Master 127.10.173.190:32997 was elected leader, sending a full tablet report...
I20260811 22:12:24.680552 20956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:24.680871 20956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:24.681957 20956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:24.683462 20956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:24.683573 20956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.683727 20956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:24.683805 20956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.720064 20956 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:33837
I20260811 22:12:24.720216 21150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:33837 every 8 connection(s)
I20260811 22:12:24.721423 20956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:24.730659 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 20956
I20260811 22:12:24.730863 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:24.731353 10934 external_mini_cluster.cc:1428] Setting key 4096d9bd0108315f4415fe2b934553ad
Aug 11 22:12:24 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486344, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:24.742307 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.724429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:57583 (local address 127.10.173.190:32997)
0811 22:12:24.724953 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:24.724958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:24.725994 (+ 1036us) server_negotiation.cc:407] Connection header received
0811 22:12:24.727892 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:24.727896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:24.727969 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:24.728075 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:24.729632 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:24.730633 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:24.732475 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:24.733052 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:24.733203 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:24.736934 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:24.736939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:24.736942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:24.736975 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:24.739717 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:24.740371 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:24.740375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:24.740377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:24.740500 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:24.740884 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:24.740888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:24.740889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:24.741146 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:24.741269 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:24.741859 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 22:12:24.742024 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":241,"threads_started":1}
I20260811 22:12:24.743434 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.725097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.131:57583)
0811 22:12:24.725812 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.725862 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:24.727592 (+ 1730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:24.728361 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:24.728375 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:24.728911 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:24.729468 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:24.729486 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:24.730829 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:24.730838 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:24.732348 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:24.732361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:24.733311 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:24.733322 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:24.733557 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:24.734310 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:24.734358 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:24.736731 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:24.739836 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:24.739843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:24.739855 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:24.740263 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:24.740597 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:24.740600 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:24.740602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:24.740787 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:24.741254 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:24.741259 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:24.741562 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 22:12:24.741795 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 22:12:24.742177 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":277,"threads_started":1}
I20260811 22:12:24.744917 21154 heartbeater.cc:347] Connected to a master server at 127.10.173.190:32997
I20260811 22:12:24.745251 21154 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:24.746008 21154 heartbeater.cc:511] Master 127.10.173.190:32997 requested a full tablet report, sending...
I20260811 22:12:24.747519 20448 ts_manager.cc:194] Registered new tserver with Master: 3d7de0600ef3442180518f144d68006b (127.10.173.131:33837)
I20260811 22:12:24.748332 20448 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:57583
I20260811 22:12:24.759377 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.173.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.189.keytab.
Entry for principal oryx/127.10.173.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.189.keytab.
I20260811 22:12:24.819147 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.189:39265
--webserver_interface=127.10.173.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.189.keytab
--principal=oryx/127.10.173.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:32997,127.10.173.189:39265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:25.172326 21166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:25.172668 21166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:25.172842 21166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:25.180263 21166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:25.180402 21166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:25.180511 21166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:25.180600 21166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:25.180671 21166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:25.190860 21166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:32997,127.10.173.189:39265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.189.keytab
--principal=oryx/127.10.173.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.189:39265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.173.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
W20260811 22:12:25.191318 21166 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.173.190:32997,127.10.173.189:39265'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 22:12:25.192430 21166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:25.193781 21166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:25.207003 21171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:25.207039 21172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:25.207217 21177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:25.208369 21166 server_base.cc:1034] running on GCE node
I20260811 22:12:25.209133 21166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:25.210638 21166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:25.210668 20943 heartbeater.cc:503] Master 127.10.173.190:32997 was elected leader, sending a full tablet report...
I20260811 22:12:25.211897 21166 hybrid_clock.cc:648] HybridClock initialized: now 1786486345211790 us; error 83 us; skew 500 ppm
I20260811 22:12:25.212330 21166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:25 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486345, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:25.216136 21166 init.cc:377] Logged in from keytab as oryx/127.10.173.189@KRBTEST.COM (short username oryx)
I20260811 22:12:25.217583 21166 webserver.cc:460] Webserver started at http://127.10.173.189:38081/ using document root <none> and password file <none>
I20260811 22:12:25.218272 21166 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:25.218364 21166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:25.218716 21166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:25.221695 21166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data/instance:
uuid: "acb8f75a203e4a578b5208089e109afc"
format_stamp: "Formatted at 2026-08-11 22:12:25 on dist-test-slave-68xk"
server_key: "1b63098aaffcfd5909be2e3daf09fc74"
server_key_iv: "9c16071f14b7364891450472ebef0a11"
server_key_version: "encryptionkey@0"
I20260811 22:12:25.222635 21166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal/instance:
uuid: "acb8f75a203e4a578b5208089e109afc"
format_stamp: "Formatted at 2026-08-11 22:12:25 on dist-test-slave-68xk"
server_key: "1b63098aaffcfd5909be2e3daf09fc74"
server_key_iv: "9c16071f14b7364891450472ebef0a11"
server_key_version: "encryptionkey@0"
I20260811 22:12:25.227918 21166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 22:12:25.231688 21187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:25.233031 21166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:12:25.233203 21166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal
uuid: "acb8f75a203e4a578b5208089e109afc"
format_stamp: "Formatted at 2026-08-11 22:12:25 on dist-test-slave-68xk"
server_key: "1b63098aaffcfd5909be2e3daf09fc74"
server_key_iv: "9c16071f14b7364891450472ebef0a11"
server_key_version: "encryptionkey@0"
I20260811 22:12:25.233423 21166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:25.249063 21166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:25.253588 21166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:25.253873 21166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:25.272900 21166 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.189:39265
I20260811 22:12:25.272924 21272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.189:39265 every 8 connection(s)
I20260811 22:12:25.274355 21166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
I20260811 22:12:25.278355 21276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:25.284138 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 21166
I20260811 22:12:25.286044 21276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:12:25 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486345, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.189@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
Aug 11 22:12:25 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.189@KRBTEST.COM
I20260811 22:12:25.307343 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.289651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36450 (local address 127.10.173.190:32997)
0811 22:12:25.290248 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:25.290263 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:25.290951 (+ 688us) server_negotiation.cc:407] Connection header received
0811 22:12:25.291302 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:25.291307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:25.291407 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:25.291544 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:25.293047 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.293823 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.295739 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.296158 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.296268 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.300477 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:25.300486 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:25.300491 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:25.300541 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:25.303406 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.304410 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.304414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.304416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.304531 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.304998 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.305003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.305005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.305478 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:25.305650 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:25.306736 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0811 22:12:25.306937 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":277,"threads_started":1}
I20260811 22:12:25.308809 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.286498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.10.173.189:39265)
0811 22:12:25.287442 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:25.287458 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:25.287515 (+ 57us) server_negotiation.cc:407] Connection header received
0811 22:12:25.290312 (+ 2797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:25.290350 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:25.290979 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:25.291326 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:25.292890 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.294506 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.295780 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.296428 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.296621 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.301147 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:25.301155 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:25.301166 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:25.301194 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:25.304817 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.305681 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.305692 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.305696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.305874 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.306268 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.306277 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.306282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.306848 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:25.307174 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:25.307827 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 22:12:25.308084 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":650,"spinlock_wait_cycles":25344,"thread_start_us":385,"threads_started":1}
I20260811 22:12:25.308849 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.289670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36450)
0811 22:12:25.290764 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.290820 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:25.291018 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:25.291630 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:25.291637 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:25.292414 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:25.292853 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.292881 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.293988 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.293994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.295581 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.295595 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.296404 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.296413 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.296619 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.297610 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:25.297633 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:25.300254 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:25.303551 (+ 3297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.303564 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.303582 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.304283 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.304646 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.304654 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.304658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.304876 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.305692 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:25.305701 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:25.306235 (+ 534us) client_negotiation.cc:769] Sending connection context
0811 22:12:25.306650 (+ 415us) client_negotiation.cc:240] Negotiation successful
0811 22:12:25.307248 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":86,"thread_start_us":548,"threads_started":1}
I20260811 22:12:25.311187 21276 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:12:25 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486345, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.189@KRBTEST.COM for oryx/127.10.173.189@KRBTEST.COM
I20260811 22:12:25.323874 21286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.312024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.0.0.1:41856)
0811 22:12:25.312197 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.312217 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:25.312329 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:25.312642 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:25.312648 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:25.313067 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:25.313198 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.313207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.314160 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.314163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.315100 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.315112 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.315723 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.315727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.315886 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.316417 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:25.316432 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:25.318372 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:25.321311 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.321317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.321321 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.321685 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.322043 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.322049 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.322052 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.322179 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.322819 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:25.322824 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:25.322972 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 22:12:25.323113 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 22:12:25.323451 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":1}
I20260811 22:12:25.323931 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.312197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.10.173.189:39265)
0811 22:12:25.312394 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:25.312398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:25.312413 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:12:25.312450 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:25.312453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:25.312503 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:25.312577 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:25.313326 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.314034 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.315224 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.315600 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.315696 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.318588 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:25.318593 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:25.318596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:25.318622 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:25.321166 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.321804 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.321811 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.321814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.321934 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.322316 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.322321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.322324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.322696 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:25.322855 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:25.323188 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 22:12:25.323582 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":197,"server-negotiator.queue_time_us":83}
I20260811 22:12:25.328675 21276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: Bootstrap starting.
I20260811 22:12:25.331977 21276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:25.333061 21276 log.cc:824] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:25.336000 21276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: No bootstrap required, opened a new log
I20260811 22:12:25.343142 21276 raft_consensus.cc:348] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } }
I20260811 22:12:25.343465 21276 raft_consensus.cc:374] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:25.343567 21276 raft_consensus.cc:729] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acb8f75a203e4a578b5208089e109afc, State: Initialized, Role: FOLLOWER
I20260811 22:12:25.344180 21276 consensus_queue.cc:260] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } }
I20260811 22:12:25.345127 21288 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } } }
I20260811 22:12:25.345417 21288 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:25.346094 21276 sys_catalog.cc:564] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:12:25.355525 21306 catalog_manager.cc:1558] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:12:25.355680 21306 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:12:25.358455 21166 master_runner.cc:428] Detected that this master acb8f75a203e4a578b5208089e109afc is joining an existing cluster
I20260811 22:12:25.358557 21166 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.173.189:39265
I20260811 22:12:25.385581 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.362448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36452 (local address 127.10.173.190:32997)
0811 22:12:25.362656 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:25.362663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:25.365324 (+ 2661us) server_negotiation.cc:407] Connection header received
0811 22:12:25.366143 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:25.366149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:25.366281 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:25.366362 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:25.368074 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.369186 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.371646 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.372355 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.372459 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.375347 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:25.375354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:25.375359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:25.375400 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:25.379154 (+ 3754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.380247 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.380264 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.380268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.380425 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.383710 (+ 3285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.383716 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.383719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.384105 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:25.384248 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:25.384891 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 22:12:25.385192 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:12:25.386902 21317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.362560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36452)
0811 22:12:25.365183 (+ 2623us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.365205 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:25.365982 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:25.366601 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:25.366606 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:25.367429 (+ 823us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:25.367883 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.367900 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.369422 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.369427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.371463 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.371479 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.372761 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.372768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.372976 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.373836 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:25.373860 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:25.375120 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:25.379331 (+ 4211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.379337 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.379340 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.380102 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.381533 (+ 1431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.381538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.381542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.381755 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.384349 (+ 2594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:25.384354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:25.384577 (+ 223us) client_negotiation.cc:769] Sending connection context
0811 22:12:25.385872 (+ 1295us) client_negotiation.cc:240] Negotiation successful
0811 22:12:25.386228 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2296,"mutex_wait_us":87,"thread_start_us":1601,"threads_started":1}
I20260811 22:12:25.387984 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.362907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.10.173.189:39265)
0811 22:12:25.363110 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:25.363118 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:25.365788 (+ 2670us) server_negotiation.cc:407] Connection header received
0811 22:12:25.365955 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:25.365962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:25.366050 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:25.366153 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:25.367837 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.369033 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.372096 (+ 3063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.373215 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.374542 (+ 1327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.379020 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:25.379029 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:25.379036 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:25.379086 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:25.383468 (+ 4382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.385384 (+ 1916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.385391 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.385394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.385576 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.386039 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.386046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.386052 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.386547 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:25.386701 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:25.387268 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 22:12:25.387526 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:12:25.389237 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.362717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.0.0.1:41870)
0811 22:12:25.365669 (+ 2952us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.365687 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:25.365825 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:25.366332 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:25.366336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:25.367160 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:25.367654 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.367673 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.369365 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.369388 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.371879 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.371896 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.375056 (+ 3160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.375065 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.375322 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.376474 (+ 1152us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:25.376500 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:25.378769 (+ 2269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:25.384572 (+ 5803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.384592 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.384597 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.385207 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.385738 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.385744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.385750 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.385910 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.386780 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:25.386787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:25.386982 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 22:12:25.388241 (+ 1259us) client_negotiation.cc:240] Negotiation successful
0811 22:12:25.388584 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2822,"mutex_wait_us":68,"thread_start_us":2407,"threads_started":1}
WARNING: no policy specified for oryx/127.10.173.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.188@KRBTEST.COM" created.
Aug 11 22:12:25 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.190@KRBTEST.COM for oryx/127.10.173.189@KRBTEST.COM
I20260811 22:12:25.426810 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.395002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.0.0.1:41876)
0811 22:12:25.395744 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.395793 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:25.396037 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:25.396649 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:25.396657 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:25.397546 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:25.397842 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.397859 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.399691 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.399697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.401361 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.401377 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.402683 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.402691 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.403098 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.404464 (+ 1366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:25.404489 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:25.411888 (+ 7399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:25.420385 (+ 8497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.420408 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.420429 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.421639 (+ 1210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.422647 (+ 1008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.422655 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.422659 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.423062 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.424306 (+ 1244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:25.424319 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:25.424766 (+ 447us) client_negotiation.cc:769] Sending connection context
0811 22:12:25.425283 (+ 517us) client_negotiation.cc:240] Negotiation successful
0811 22:12:25.425897 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":296,"threads_started":1}
I20260811 22:12:25.427567 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.394677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41876 (local address 127.10.173.189:39265)
0811 22:12:25.394853 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:25.394859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:25.395880 (+ 1021us) server_negotiation.cc:407] Connection header received
0811 22:12:25.396288 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:25.396298 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:25.396443 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:25.396600 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:25.398068 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.399528 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.401534 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.402472 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.402645 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.412383 (+ 9738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:25.412406 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:25.412426 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:25.412530 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:25.420059 (+ 7529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.421884 (+ 1825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.421906 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.421913 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.422284 (+ 371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.423247 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.423257 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.423275 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.424017 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:25.424292 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:25.425471 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0811 22:12:25.426054 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:12:25.432058 20448 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.173.189:39265: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "acb8f75a203e4a578b5208089e109afc"
member_type: NON_VOTER
last_known_addr {
host: "127.10.173.189"
port: 39265
}
attrs {
promote: true
}
}
dest_uuid: "11e46714a91e4356a74769c0ceaaed9c"
cas_config_opid_index: -1
I20260811 22:12:25.435168 20448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.188.keytab.
Entry for principal oryx/127.10.173.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.188.keytab.
W20260811 22:12:25.474537 10934 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.10.173.188: real 0.108s user 0.006s sys 0.029s
I20260811 22:12:25.477173 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.188:37661
--webserver_interface=127.10.173.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.188.keytab
--principal=oryx/127.10.173.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:32997,127.10.173.189:39265,127.10.173.188:37661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:25.478255 21234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:25.475299 21326 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER acb8f75a203e4a578b5208089e109afc (127.10.173.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } }
I20260811 22:12:25.486637 21234 raft_consensus.cc:1261] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Refusing update from remote peer 11e46714a91e4356a74769c0ceaaed9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 22:12:25.493700 21327 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:25.494849 21326 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.173.189:39265
I20260811 22:12:25.496817 21166 master.cc:537] Master@127.10.173.189:39265 shutting down...
W20260811 22:12:25.506237 21196 tablet.cc:2369] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:25.511806 21328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } }
I20260811 22:12:25.512645 21328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
W20260811 22:12:25.518500 20421 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 22:12:25.552416 21166 raft_consensus.cc:2229] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:25.553236 21166 raft_consensus.cc:2258] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:25.553603 21166 tablet_replica.cc:330] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: stopping tablet replica
W20260811 22:12:25.557682 20421 consensus_peers.cc:487] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c -> Peer acb8f75a203e4a578b5208089e109afc (127.10.173.189:39265): Couldn't send request to peer acb8f75a203e4a578b5208089e109afc. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:12:25.571556 21166 master.cc:552] Master@127.10.173.189:39265 shutdown complete.
I20260811 22:12:25.571791 21166 master_runner.cc:305] Clearing existing system tablet
I20260811 22:12:25.572640 21166 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:25.572815 21166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:25.587675 21166 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.001s sys 0.005s
I20260811 22:12:25.607719 21340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:25.611614 21166 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:25.612583 21166 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.008s sys 0.012s
I20260811 22:12:25.612902 21166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal
uuid: "acb8f75a203e4a578b5208089e109afc"
format_stamp: "Formatted at 2026-08-11 22:12:25 on dist-test-slave-68xk"
server_key: "1b63098aaffcfd5909be2e3daf09fc74"
server_key_iv: "9c16071f14b7364891450472ebef0a11"
server_key_version: "encryptionkey@0"
I20260811 22:12:25.623448 21166 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:25.638350 21166 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:12:25.638916 21166 log.cc:1196] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 22:12:25.639951 21166 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: Deleting consensus metadata
I20260811 22:12:25.640558 21166 master_runner.cc:315] Copying system tablet from 127.10.173.190:32997
I20260811 22:12:25.705582 21166 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: tablet copy: Beginning tablet copy session from remote peer at address 127.10.173.190:32997
I20260811 22:12:25.750882 21154 heartbeater.cc:503] Master 127.10.173.190:32997 was elected leader, sending a full tablet report...
I20260811 22:12:25.760147 21284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.708825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36468 (local address 127.10.173.190:32997)
0811 22:12:25.709139 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:25.709147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:25.709312 (+ 165us) server_negotiation.cc:407] Connection header received
0811 22:12:25.709892 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:25.709898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:25.710063 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:25.710245 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:25.712959 (+ 2714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.714965 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.718464 (+ 3499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.719790 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.719990 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.738895 (+ 18905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:25.738928 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:25.738945 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:25.739057 (+ 112us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:25.750393 (+ 11336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.752165 (+ 1772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.752187 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.752205 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.754714 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.755863 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.755871 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.755877 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.756785 (+ 908us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:25.757131 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:25.758498 (+ 1367us) server_negotiation.cc:299] Negotiation successful
0811 22:12:25.759053 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:12:25.762964 21350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.707907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36468)
0811 22:12:25.709086 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.709124 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:25.709526 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:25.710749 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:25.710759 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:25.711995 (+ 1236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:25.712591 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.712620 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.715401 (+ 2781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.715408 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.718059 (+ 2651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.718092 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.731071 (+ 12979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.731093 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.731574 (+ 481us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.734724 (+ 3150us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:25.734782 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:25.737052 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:25.750793 (+ 13741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.750812 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.750815 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.751819 (+ 1004us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.755072 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.755082 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.755100 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.755512 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.757327 (+ 1815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:25.757336 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:25.757724 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 22:12:25.761120 (+ 3396us) client_negotiation.cc:240] Negotiation successful
0811 22:12:25.761802 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"thread_start_us":991,"threads_started":1}
I20260811 22:12:25.764905 20477 tablet_copy_service.cc:140] P 11e46714a91e4356a74769c0ceaaed9c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer acb8f75a203e4a578b5208089e109afc ({username='oryx', principal='oryx/127.10.173.189@KRBTEST.COM'} at 127.0.0.1:36468)
I20260811 22:12:25.765601 20477 tablet_copy_service.cc:161] P 11e46714a91e4356a74769c0ceaaed9c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer acb8f75a203e4a578b5208089e109afc at {username='oryx', principal='oryx/127.10.173.189@KRBTEST.COM'} at 127.0.0.1:36468: session id = acb8f75a203e4a578b5208089e109afc-00000000000000000000000000000000
I20260811 22:12:25.775257 20477 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:12:25.780982 21166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:12:25.797549 21166 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: tablet copy: Starting download of 0 data blocks...
I20260811 22:12:25.798679 21166 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: tablet copy: Starting download of 1 WAL segments...
I20260811 22:12:25.805939 21166 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:12:25.816144 20477 tablet_copy_service.cc:342] P 11e46714a91e4356a74769c0ceaaed9c: Request end of tablet copy session acb8f75a203e4a578b5208089e109afc-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.173.189@KRBTEST.COM'} at 127.0.0.1:36468
I20260811 22:12:25.816700 20477 tablet_copy_service.cc:433] P 11e46714a91e4356a74769c0ceaaed9c: ending tablet copy session acb8f75a203e4a578b5208089e109afc-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer acb8f75a203e4a578b5208089e109afc
I20260811 22:12:25.848078 21166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:25.865455 21351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:25.877578 21357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:25.878579 21166 server_base.cc:1034] running on GCE node
W20260811 22:12:25.879066 21353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:25.880735 21166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:25.890625 21166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:25.894153 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.893051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.0.0.1:41884)
0811 22:12:25.893514 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.893798 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:39265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261}
I20260811 22:12:25.896384 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.895605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.0.0.1:41888)
0811 22:12:25.895943 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.896113 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:39265: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260811 22:12:25.899657 21166 hybrid_clock.cc:648] HybridClock initialized: now 1786486345899542 us; error 89 us; skew 500 ppm
I20260811 22:12:25.900105 21166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:25 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486345, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:25.904028 21166 init.cc:377] Logged in from keytab as oryx/127.10.173.189@KRBTEST.COM (short username oryx)
I20260811 22:12:25.937268 21166 webserver.cc:460] Webserver started at http://127.10.173.189:36417/ using document root <none> and password file <none>
I20260811 22:12:25.938263 21166 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:25.938611 21166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:25.951721 21166 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.002s sys 0.009s
I20260811 22:12:25.963830 21368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:25.965547 21166 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.005s sys 0.005s
I20260811 22:12:25.965922 21166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal
uuid: "acb8f75a203e4a578b5208089e109afc"
format_stamp: "Formatted at 2026-08-11 22:12:25 on dist-test-slave-68xk"
server_key: "1b63098aaffcfd5909be2e3daf09fc74"
server_key_iv: "9c16071f14b7364891450472ebef0a11"
server_key_version: "encryptionkey@0"
I20260811 22:12:25.966336 21166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:26.076691 21166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:26.089852 21166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:26.209102 21166 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.189:39265
I20260811 22:12:26.209299 21452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.189:39265 every 8 connection(s)
I20260811 22:12:26.211583 21166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
I20260811 22:12:26.222903 21453 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:12:26.230185 21453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: Bootstrap starting.
I20260811 22:12:26.269151 21453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:12:26.270383 21453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: Bootstrap complete.
I20260811 22:12:26.272770 21453 raft_consensus.cc:348] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } }
I20260811 22:12:26.273278 21453 raft_consensus.cc:729] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: acb8f75a203e4a578b5208089e109afc, State: Initialized, Role: LEARNER
I20260811 22:12:26.273797 21453 consensus_queue.cc:260] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } }
I20260811 22:12:26.275602 21459 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } }
I20260811 22:12:26.276315 21453 sys_catalog.cc:564] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:26.276327 21459 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: This master's current role is: LEARNER
I20260811 22:12:26.296139 21477 catalog_manager.cc:1259] Loaded cluster ID: 35427e1bb15c4054a01441bf811a0e08
I20260811 22:12:26.296334 21477 catalog_manager.cc:1552] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: loading cluster ID for follower catalog manager: success
I20260811 22:12:26.303624 21477 catalog_manager.cc:1574] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: acquiring CA information for follower catalog manager: success
I20260811 22:12:26.308667 21477 catalog_manager.cc:1602] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:12:26.400633 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.383812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.0.0.1:41900)
0811 22:12:26.384394 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.384453 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:26.384753 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:26.387233 (+ 2480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:26.387241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:26.387334 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:26.387753 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.387789 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.391655 (+ 3866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.391662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.395297 (+ 3635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.395744 (+ 447us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.399112 (+ 3368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.399127 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.399515 (+ 388us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.399659 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 22:12:26.399860 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 22:12:26.400120 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313}
I20260811 22:12:26.400863 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.384173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.10.173.189:39265)
0811 22:12:26.386170 (+ 1997us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:26.386192 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:26.386239 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:12:26.386548 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:26.386570 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:26.386806 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:26.388156 (+ 1350us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:26.388597 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.391309 (+ 2712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.396014 (+ 4705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.398791 (+ 2777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.399169 (+ 378us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.399576 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:26.399932 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 22:12:26.400190 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1680,"thread_start_us":1200,"threads_started":1}
I20260811 22:12:26.403539 21416 raft_consensus.cc:1203] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 22:12:26.404716 21459 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11e46714a91e4356a74769c0ceaaed9c. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } }
I20260811 22:12:26.405231 21459 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: This master's current role is: LEARNER
W20260811 22:12:26.442243 21329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:26.442833 21329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:26.443084 21329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:26.458009 21329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:26.458175 21329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:26.458318 21329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:26.458513 21329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:26.458631 21329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:26.478964 21329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:32997,127.10.173.189:39265,127.10.173.188:37661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.188.keytab
--principal=oryx/127.10.173.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.188:37661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.173.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:26.481146 21329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:26.483438 21329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:26.508826 21486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:26.508893 21487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:26.510218 21491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:26.512674 21329 server_base.cc:1034] running on GCE node
I20260811 22:12:26.514753 21329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:26.518621 21329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:26.520138 21329 hybrid_clock.cc:648] HybridClock initialized: now 1786486346519993 us; error 96 us; skew 500 ppm
I20260811 22:12:26.521004 21329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:26 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486346, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:26.529074 21329 init.cc:377] Logged in from keytab as oryx/127.10.173.188@KRBTEST.COM (short username oryx)
I20260811 22:12:26.532356 21329 webserver.cc:460] Webserver started at http://127.10.173.188:45805/ using document root <none> and password file <none>
I20260811 22:12:26.533855 21329 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:26.534200 21329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:26.534914 21329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:26.540675 21329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data/instance:
uuid: "ff65fc83f0854276a28f78343c6d7d31"
format_stamp: "Formatted at 2026-08-11 22:12:26 on dist-test-slave-68xk"
server_key: "64c1b51471eab69f52ef7a95af07b462"
server_key_iv: "e3c08027fee40c1ef1255c0d6e9d82be"
server_key_version: "encryptionkey@0"
I20260811 22:12:26.542093 21329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal/instance:
uuid: "ff65fc83f0854276a28f78343c6d7d31"
format_stamp: "Formatted at 2026-08-11 22:12:26 on dist-test-slave-68xk"
server_key: "64c1b51471eab69f52ef7a95af07b462"
server_key_iv: "e3c08027fee40c1ef1255c0d6e9d82be"
server_key_version: "encryptionkey@0"
I20260811 22:12:26.553319 21329 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.004s
I20260811 22:12:26.561519 21504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:26.564497 21329 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260811 22:12:26.564889 21329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal
uuid: "ff65fc83f0854276a28f78343c6d7d31"
format_stamp: "Formatted at 2026-08-11 22:12:26 on dist-test-slave-68xk"
server_key: "64c1b51471eab69f52ef7a95af07b462"
server_key_iv: "e3c08027fee40c1ef1255c0d6e9d82be"
server_key_version: "encryptionkey@0"
I20260811 22:12:26.565335 21329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:26.588917 21329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:26.596918 21329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:26.597525 21329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:26.640506 21329 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.188:37661
I20260811 22:12:26.640507 21584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.188:37661 every 8 connection(s)
I20260811 22:12:26.643836 21329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
I20260811 22:12:26.649387 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 21329
I20260811 22:12:26.651532 21585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:26.666527 21585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:12:26 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.188@KRBTEST.COM
Aug 11 22:12:26 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486346, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.188@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:26.692883 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.653133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41816 (local address 127.10.173.188:37661)
0811 22:12:26.654404 (+ 1271us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:26.654494 (+ 90us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:26.654569 (+ 75us) server_negotiation.cc:407] Connection header received
0811 22:12:26.657149 (+ 2580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:26.657173 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:26.657769 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:26.658389 (+ 620us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:26.661072 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.663442 (+ 2370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.665600 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.666985 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.667355 (+ 370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.677138 (+ 9783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:26.677153 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:26.677191 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:26.677302 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:26.684223 (+ 6921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.685546 (+ 1323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.685569 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.685602 (+ 33us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.685873 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.686644 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.686659 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.686663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.687739 (+ 1076us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:26.688403 (+ 664us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:26.689351 (+ 948us) server_negotiation.cc:299] Negotiation successful
0811 22:12:26.690290 (+ 939us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":947,"thread_start_us":572,"threads_started":1}
I20260811 22:12:26.703184 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.672354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.10.173.190:32997)
0811 22:12:26.673347 (+ 993us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:26.673354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:26.674980 (+ 1626us) server_negotiation.cc:407] Connection header received
0811 22:12:26.675356 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:26.675366 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:26.675535 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:26.675824 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:26.678406 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.680131 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.683743 (+ 3612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.684421 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.684662 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.692773 (+ 8111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:26.692780 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:26.692783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:26.692837 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:26.698332 (+ 5495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.699563 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.699572 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.699587 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.699781 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.700457 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.700464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.700469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.700935 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:26.701233 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:26.701976 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 22:12:26.702444 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":549,"threads_started":1}
I20260811 22:12:26.703227 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.672411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36482)
0811 22:12:26.674706 (+ 2295us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.674798 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:26.675098 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:26.675886 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:26.675892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:26.677337 (+ 1445us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:26.678151 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.678169 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.680369 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.680376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.683427 (+ 3051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.683449 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.684734 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.684744 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.685110 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.686028 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:26.686060 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:26.692433 (+ 6373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:26.698669 (+ 6236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.698689 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.698694 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.699339 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.700009 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.700017 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.700031 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.700256 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.701179 (+ 923us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:26.701187 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:26.701475 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 22:12:26.701790 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 22:12:26.702443 (+ 653us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1823,"thread_start_us":1479,"threads_started":1}
I20260811 22:12:26.705767 21585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:12:26 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486346, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.188@KRBTEST.COM for oryx/127.10.173.189@KRBTEST.COM
I20260811 22:12:26.729018 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.706976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41916 (local address 127.10.173.189:39265)
0811 22:12:26.707347 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:26.707364 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:26.707484 (+ 120us) server_negotiation.cc:407] Connection header received
0811 22:12:26.707859 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:26.707865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:26.707945 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:26.708062 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:26.709265 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.710351 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.712341 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.712974 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.713258 (+ 284us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.718305 (+ 5047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:26.718321 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:26.718324 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:26.718371 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:26.723666 (+ 5295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.724939 (+ 1273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.724952 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.724956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.725211 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.725949 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.725968 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.725974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.726611 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:26.726929 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:26.727635 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 22:12:26.728246 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:12:26.729004 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.706817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.0.0.1:41916)
0811 22:12:26.707303 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.707345 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:26.707607 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:26.708093 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:26.708097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:26.708798 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:26.709028 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.709052 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.710612 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.710622 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.712141 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.712161 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.713275 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.713284 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.713632 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.714578 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:26.714612 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:26.717966 (+ 3354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:26.723906 (+ 5940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.723916 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.723932 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.724679 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.725462 (+ 783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.725469 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.725484 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.725718 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.726830 (+ 1112us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:26.726837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:26.727149 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 22:12:26.727433 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 22:12:26.728179 (+ 746us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 22:12:26.730983 21585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:12:26 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486346, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.188@KRBTEST.COM for oryx/127.10.173.188@KRBTEST.COM
I20260811 22:12:26.753840 21590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.731930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37661 (local address 127.0.0.1:41820)
0811 22:12:26.732264 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.732295 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:26.732463 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:26.732974 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:26.732981 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:26.733686 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:26.733893 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.733911 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.735471 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.735476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.737314 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.737338 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.738651 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.738666 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.738961 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.739998 (+ 1037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:26.740088 (+ 90us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:26.743842 (+ 3754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:26.749374 (+ 5532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.749379 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.749389 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.750506 (+ 1117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.751298 (+ 792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.751303 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.751306 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.751574 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.752497 (+ 923us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:26.752500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:26.752743 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 22:12:26.752967 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 22:12:26.753306 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:12:26.754339 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.732301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41820 (local address 127.10.173.188:37661)
0811 22:12:26.732518 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:26.732522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:26.732549 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:12:26.732599 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:26.732603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:26.732686 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:26.732909 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:26.734068 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.735278 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.737551 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.738353 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.738581 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.744095 (+ 5514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:26.744101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:26.744105 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:26.744154 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:26.749102 (+ 4948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.750753 (+ 1651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.750771 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.750775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.751096 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.751764 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.751770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.751773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.752329 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:26.752529 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:26.753228 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 22:12:26.753626 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 22:12:26.763726 21585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Bootstrap starting.
I20260811 22:12:26.769246 21585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:26.770720 21585 log.cc:824] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:26.774716 21585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: No bootstrap required, opened a new log
I20260811 22:12:26.787932 21585 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } }
I20260811 22:12:26.788508 21585 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:26.788666 21585 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff65fc83f0854276a28f78343c6d7d31, State: Initialized, Role: FOLLOWER
I20260811 22:12:26.789427 21585 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } }
I20260811 22:12:26.790899 21594 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } } }
I20260811 22:12:26.791379 21594 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:26.792577 21585 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:26.803728 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 20539
W20260811 22:12:26.808789 21613 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:12:26.809029 21613 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:12:26.815143 21329 master_runner.cc:428] Detected that this master ff65fc83f0854276a28f78343c6d7d31 is joining an existing cluster
I20260811 22:12:26.815490 21329 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.173.188:37661
Aug 11 22:12:26 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.129@KRBTEST.COM
I20260811 22:12:26.847913 21617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.806720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58806 (local address 127.10.173.129:45861)
0811 22:12:26.809897 (+ 3177us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:26.809919 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:26.809974 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:12:26.810101 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:26.810118 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:26.810488 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:26.810677 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:26.813392 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.815115 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.817778 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.819671 (+ 1893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.819911 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.829224 (+ 9313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:26.829254 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:26.829282 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:26.829352 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:26.837015 (+ 7663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.839338 (+ 2323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.839376 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.839380 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.839842 (+ 462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.841770 (+ 1928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.841782 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.841801 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.842960 (+ 1159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:26.845727 (+ 2767us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:26.846040 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 22:12:26.846574 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2917,"thread_start_us":2693,"threads_started":1}
I20260811 22:12:26.851462 20731 generic_service.cc:196] Checking for leaks (request via RPC)
W20260811 22:12:26.852316 21512 tablet.cc:2369] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:26.863396 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.827505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36494 (local address 127.10.173.190:32997)
0811 22:12:26.827876 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:26.827896 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:26.829698 (+ 1802us) server_negotiation.cc:407] Connection header received
0811 22:12:26.830117 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:26.830123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:26.830219 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:26.830305 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:26.832786 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.834806 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.838081 (+ 3275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.839403 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.839587 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.844264 (+ 4677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:26.844274 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:26.844287 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:26.844359 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:26.855577 (+ 11218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.857109 (+ 1532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.857129 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.857133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.857433 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.859477 (+ 2044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.859487 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.859492 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.860203 (+ 711us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:26.860398 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:26.861969 (+ 1571us) server_negotiation.cc:299] Negotiation successful
0811 22:12:26.862472 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:12:26.868422 21625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.822592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36494)
0811 22:12:26.829567 (+ 6975us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.829593 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:26.829889 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:26.830564 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:26.830570 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:26.831525 (+ 955us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:26.832344 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.832369 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.835027 (+ 2658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.835033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.837779 (+ 2746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.837805 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.840114 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.840124 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.840462 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.841752 (+ 1290us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:26.841820 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:26.843920 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:26.855917 (+ 11997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.855922 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.855925 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.856826 (+ 901us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.857712 (+ 886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.857720 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.857725 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.859186 (+ 1461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.860588 (+ 1402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:26.860594 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:26.861290 (+ 696us) client_negotiation.cc:769] Sending connection context
0811 22:12:26.863985 (+ 2695us) client_negotiation.cc:240] Negotiation successful
0811 22:12:26.866147 (+ 2162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6802,"thread_start_us":9495,"threads_started":1}
I20260811 22:12:26.869089 21594 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:26.869871 21594 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } }
I20260811 22:12:26.875213 20462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff65fc83f0854276a28f78343c6d7d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11e46714a91e4356a74769c0ceaaed9c" is_pre_election: true
I20260811 22:12:26.876838 21416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff65fc83f0854276a28f78343c6d7d31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acb8f75a203e4a578b5208089e109afc" is_pre_election: true
I20260811 22:12:26.878924 21505 leader_election.cc:302] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff65fc83f0854276a28f78343c6d7d31; no voters: 11e46714a91e4356a74769c0ceaaed9c, acb8f75a203e4a578b5208089e109afc
I20260811 22:12:26.875052 21594 leader_election.cc:288] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997), acb8f75a203e4a578b5208089e109afc (127.10.173.189:39265)
I20260811 22:12:26.883639 21594 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 22:12:26.891409 21478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.823074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41930 (local address 127.10.173.189:39265)
0811 22:12:26.823501 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:26.823514 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:26.834149 (+ 10635us) server_negotiation.cc:407] Connection header received
0811 22:12:26.834710 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:26.834719 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:26.834811 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:26.834911 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:26.841920 (+ 7009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.844264 (+ 2344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.850963 (+ 6699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.852518 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.852728 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.863326 (+ 10598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:26.863350 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:26.863371 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:26.863484 (+ 113us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:26.871354 (+ 7870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.881024 (+ 9670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.881051 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.881058 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.881825 (+ 767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.883143 (+ 1318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.883155 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.883172 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.884334 (+ 1162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:26.884673 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:26.889731 (+ 5058us) server_negotiation.cc:299] Negotiation successful
0811 22:12:26.890318 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260811 22:12:26.893971 21626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.824334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.0.0.1:41930)
0811 22:12:26.833957 (+ 9623us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.833994 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:26.834383 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:26.835301 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:26.835309 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:26.838874 (+ 3565us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:26.839840 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.839858 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.844728 (+ 4870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.844736 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.850340 (+ 5604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.850371 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.853518 (+ 3147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.853540 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.854626 (+ 1086us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.859106 (+ 4480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:26.859166 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:26.862111 (+ 2945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:26.871826 (+ 9715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.871833 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.871847 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.872923 (+ 1076us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.882206 (+ 9283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.882213 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.882218 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.882702 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.884915 (+ 2213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:26.884934 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:26.885523 (+ 589us) client_negotiation.cc:769] Sending connection context
0811 22:12:26.891968 (+ 6445us) client_negotiation.cc:240] Negotiation successful
0811 22:12:26.892684 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9375,"mutex_wait_us":326,"thread_start_us":7228,"threads_started":1}
I20260811 22:12:26.915881 21588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.833155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41826 (local address 127.10.173.188:37661)
0811 22:12:26.833515 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:26.833533 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:26.850026 (+ 16493us) server_negotiation.cc:407] Connection header received
0811 22:12:26.850627 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:26.850633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:26.850811 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:26.851051 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:26.855878 (+ 4827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.857587 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.873624 (+ 16037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.875006 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.875236 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.899405 (+ 24169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:26.899414 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:26.899420 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:26.899508 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:26.908863 (+ 9355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.910870 (+ 2007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.910880 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.910884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.911205 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.912100 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.912120 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.912124 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.912947 (+ 823us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:26.913186 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:26.914524 (+ 1338us) server_negotiation.cc:299] Negotiation successful
0811 22:12:26.915029 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260811 22:12:26.918128 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.832699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37661 (local address 127.0.0.1:41826)
0811 22:12:26.849842 (+ 17143us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.849895 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:26.850201 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:26.851500 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:26.851506 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:26.853456 (+ 1950us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:26.854761 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.854793 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.858283 (+ 3490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.858293 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.873345 (+ 15052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.873377 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.887741 (+ 14364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.887763 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.888212 (+ 449us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.890299 (+ 2087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:26.890398 (+ 99us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:26.898817 (+ 8419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:26.909211 (+ 10394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.909230 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.909235 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.910578 (+ 1343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.911517 (+ 939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.911524 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.911540 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.911845 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.913370 (+ 1525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:26.913397 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:26.913729 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 22:12:26.916480 (+ 2751us) client_negotiation.cc:240] Negotiation successful
0811 22:12:26.917122 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16810,"mutex_wait_us":1,"thread_start_us":17864,"threads_started":1}
Aug 11 22:12:26 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.190@KRBTEST.COM for oryx/127.10.173.188@KRBTEST.COM
I20260811 22:12:26.935967 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.885994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.10.173.188:37661)
0811 22:12:26.898179 (+ 12185us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:26.898185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:26.898231 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:12:26.898384 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:26.898389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:26.898602 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:26.900398 (+ 1796us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:26.901014 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.906300 (+ 5286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.909744 (+ 3444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.911386 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.921276 (+ 9890us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.921705 (+ 429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:26.921712 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:26.921717 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:26.921784 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:26.928573 (+ 6789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.930404 (+ 1831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.930441 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.930445 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.930775 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:26.931527 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:26.931547 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:26.931551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:26.932614 (+ 1063us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:26.932885 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:26.934689 (+ 1804us) server_negotiation.cc:299] Negotiation successful
0811 22:12:26.935080 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11901,"thread_start_us":18148,"threads_started":1}
I20260811 22:12:26.937968 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.885825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37661 (local address 127.0.0.1:41834)
0811 22:12:26.886920 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.886962 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:26.887209 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:26.898960 (+ 11751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:26.898967 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:26.899768 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:26.900120 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.900147 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.906699 (+ 6552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.906708 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.909341 (+ 2633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:26.909372 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:26.911905 (+ 2533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:26.911915 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:26.912377 (+ 462us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:26.913847 (+ 1470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:26.913908 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:26.920907 (+ 6999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:26.928871 (+ 7964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.928891 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.928897 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.930065 (+ 1168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.931021 (+ 956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:26.931026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:26.931040 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:26.931327 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:26.932829 (+ 1502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:26.932844 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:26.933133 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 22:12:26.936830 (+ 3697us) client_negotiation.cc:240] Negotiation successful
0811 22:12:26.937315 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745}
I20260811 22:12:26.940943 20448 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.173.188:37661: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31"
member_type: NON_VOTER
last_known_addr {
host: "127.10.173.188"
port: 37661
}
attrs {
promote: true
}
}
dest_uuid: "11e46714a91e4356a74769c0ceaaed9c"
cas_config_opid_index: 5
I20260811 22:12:26.943543 20448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } }
I20260811 22:12:26.956338 21633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } } pending_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:26.957352 21633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:26.959589 21637 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ff65fc83f0854276a28f78343c6d7d31 (127.10.173.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:26.962507 21552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:26.966025 21637 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.173.188:37661
I20260811 22:12:26.967391 21416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 LEARNER]: Refusing update from remote peer 11e46714a91e4356a74769c0ceaaed9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:12:26.972886 21552 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 FOLLOWER]: Refusing update from remote peer 11e46714a91e4356a74769c0ceaaed9c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:12:26.974992 21633 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 22:12:26.975934 21633 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:26.990857 21329 master.cc:537] Master@127.10.173.188:37661 shutting down...
I20260811 22:12:26.998559 21639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:26.999109 21639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:27.009640 21329 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:27.010149 21329 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:27.013185 21329 tablet_replica.cc:330] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: stopping tablet replica
I20260811 22:12:27.023931 21329 master.cc:552] Master@127.10.173.188:37661 shutdown complete.
I20260811 22:12:27.024063 21329 master_runner.cc:305] Clearing existing system tablet
I20260811 22:12:27.024592 21329 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:27.024725 21329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:27.028548 21416 raft_consensus.cc:2932] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ff65fc83f0854276a28f78343c6d7d31 (127.10.173.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:27.030789 21329 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.002s
W20260811 22:12:27.039660 20421 consensus_peers.cc:487] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c -> Peer ff65fc83f0854276a28f78343c6d7d31 (127.10.173.188:37661): Couldn't send request to peer ff65fc83f0854276a28f78343c6d7d31. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:12:27.044268 21639 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c: attempting to promote NON_VOTER acb8f75a203e4a578b5208089e109afc to VOTER
I20260811 22:12:27.046017 21639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } }
I20260811 22:12:27.049604 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.048922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37661 (local address 127.0.0.1:41840)
0811 22:12:27.049165 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:27.049382 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 22:12:27.051192 21415 raft_consensus.cc:1261] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 LEARNER]: Refusing update from remote peer 11e46714a91e4356a74769c0ceaaed9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 22:12:27.052256 21646 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 22:12:27.054601 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.053734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37661 (local address 127.0.0.1:41844)
0811 22:12:27.053928 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:27.054033 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:12:27.056490 20421 consensus_peers.cc:487] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c -> Peer ff65fc83f0854276a28f78343c6d7d31 (127.10.173.188:37661): Couldn't send request to peer ff65fc83f0854276a28f78343c6d7d31. Status: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37661: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:12:27.057559 21655 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: NON_VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: true } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:27.058482 21655 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:27.059188 21633 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, acb8f75a203e4a578b5208089e109afc (127.10.173.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:27.069717 21639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:27.070237 21639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:27.071440 21646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:27.071908 21646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:27.073145 21415 raft_consensus.cc:2932] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, acb8f75a203e4a578b5208089e109afc (127.10.173.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:27.077121 21653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:27.078827 21655 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:27.079118 21655 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:27.080422 21329 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:27.080807 21329 fs_manager.cc:725] Time spent opening block manager: real 0.049s user 0.015s sys 0.025s
I20260811 22:12:27.080948 21329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal
uuid: "ff65fc83f0854276a28f78343c6d7d31"
format_stamp: "Formatted at 2026-08-11 22:12:26 on dist-test-slave-68xk"
server_key: "64c1b51471eab69f52ef7a95af07b462"
server_key_iv: "e3c08027fee40c1ef1255c0d6e9d82be"
server_key_version: "encryptionkey@0"
I20260811 22:12:27.081574 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 20539
I20260811 22:12:27.087955 21329 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:27.097790 21329 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:12:27.097988 21329 log.cc:1196] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 22:12:27.098634 21329 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Deleting consensus metadata
I20260811 22:12:27.098964 21329 master_runner.cc:315] Copying system tablet from 127.10.173.190:32997
I20260811 22:12:27.102825 21329 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: tablet copy: Beginning tablet copy session from remote peer at address 127.10.173.190:32997
I20260811 22:12:27.118266 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:45861
--local_ip_for_outbound_sockets=127.10.173.129
--tserver_master_addrs=127.10.173.190:32997
--webserver_port=39649
--webserver_interface=127.10.173.129
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab
--principal=oryx/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.173.190:32997,127.10.173.189:39265
--tserver_master_addrs=127.10.173.190:32997,127.10.173.189:39265,127.10.173.188:37661 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:27.121618 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.103936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36506 (local address 127.10.173.190:32997)
0811 22:12:27.104200 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:27.104209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:27.104679 (+ 470us) server_negotiation.cc:407] Connection header received
0811 22:12:27.104903 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:27.104911 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:27.105015 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:27.105146 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:27.106510 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.107754 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.110206 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.111102 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.111249 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.113934 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:27.113945 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:27.113954 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:27.114003 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:27.118241 (+ 4238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:27.119284 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:27.119291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:27.119295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:27.119443 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:27.119919 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:27.119926 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:27.119930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:27.120226 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:27.120395 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:27.120881 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 22:12:27.121114 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 22:12:27.121659 21665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.103791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36506)
0811 22:12:27.104529 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:27.104561 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:27.104745 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:27.105209 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:27.105218 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:27.105904 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:27.106286 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:27.106307 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.107918 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.107931 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:27.110017 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:27.110038 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.111298 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.111308 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:27.111604 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.112398 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:27.112428 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:27.113717 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:27.118704 (+ 4987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:27.118710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:27.118716 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:27.119138 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:27.119588 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:27.119594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:27.119598 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:27.119792 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:27.120376 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:27.120381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:27.120568 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 22:12:27.120764 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:12:27.121133 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":358,"threads_started":1}
I20260811 22:12:27.123457 20477 tablet_copy_service.cc:140] P 11e46714a91e4356a74769c0ceaaed9c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ff65fc83f0854276a28f78343c6d7d31 ({username='oryx', principal='oryx/127.10.173.188@KRBTEST.COM'} at 127.0.0.1:36506)
I20260811 22:12:27.123656 20477 tablet_copy_service.cc:161] P 11e46714a91e4356a74769c0ceaaed9c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ff65fc83f0854276a28f78343c6d7d31 at {username='oryx', principal='oryx/127.10.173.188@KRBTEST.COM'} at 127.0.0.1:36506: session id = ff65fc83f0854276a28f78343c6d7d31-00000000000000000000000000000000
I20260811 22:12:27.127733 20477 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:12:27.129645 21329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:12:27.137616 21329 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: tablet copy: Starting download of 0 data blocks...
I20260811 22:12:27.138079 21329 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: tablet copy: Starting download of 1 WAL segments...
I20260811 22:12:27.141067 21329 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:12:27.146315 20477 tablet_copy_service.cc:342] P 11e46714a91e4356a74769c0ceaaed9c: Request end of tablet copy session ff65fc83f0854276a28f78343c6d7d31-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.173.188@KRBTEST.COM'} at 127.0.0.1:36506
I20260811 22:12:27.146498 20477 tablet_copy_service.cc:433] P 11e46714a91e4356a74769c0ceaaed9c: ending tablet copy session ff65fc83f0854276a28f78343c6d7d31-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ff65fc83f0854276a28f78343c6d7d31
I20260811 22:12:27.157819 21329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:27.160713 21667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:27.174796 21668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:27.178742 21670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:27.187952 21329 server_base.cc:1034] running on GCE node
I20260811 22:12:27.188474 21329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:27.189286 21329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:27.190508 21329 hybrid_clock.cc:648] HybridClock initialized: now 1786486347190456 us; error 42 us; skew 500 ppm
I20260811 22:12:27.190798 21329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:27 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486347, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:27.192981 21329 init.cc:377] Logged in from keytab as oryx/127.10.173.188@KRBTEST.COM (short username oryx)
I20260811 22:12:27.195425 21329 webserver.cc:460] Webserver started at http://127.10.173.188:32897/ using document root <none> and password file <none>
I20260811 22:12:27.195822 21329 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:27.195946 21329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:27.198596 21329 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:12:27.200470 21679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:27.201020 21329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 22:12:27.201162 21329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal
uuid: "ff65fc83f0854276a28f78343c6d7d31"
format_stamp: "Formatted at 2026-08-11 22:12:26 on dist-test-slave-68xk"
server_key: "64c1b51471eab69f52ef7a95af07b462"
server_key_iv: "e3c08027fee40c1ef1255c0d6e9d82be"
server_key_version: "encryptionkey@0"
I20260811 22:12:27.201378 21329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:27.239279 21329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:27.243543 21329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:27.275257 21329 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.188:37661
I20260811 22:12:27.275288 21755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.188:37661 every 8 connection(s)
I20260811 22:12:27.276450 21329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
I20260811 22:12:27.286897 21758 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:12:27.289786 21758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Bootstrap starting.
I20260811 22:12:27.305965 21758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:12:27.306605 21758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Bootstrap complete.
I20260811 22:12:27.307610 21758 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } }
I20260811 22:12:27.307904 21758 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ff65fc83f0854276a28f78343c6d7d31, State: Initialized, Role: LEARNER
I20260811 22:12:27.308213 21758 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } }
I20260811 22:12:27.309511 21758 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:27.309921 21762 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:27.310475 21762 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:12:27.320152 21777 catalog_manager.cc:1259] Loaded cluster ID: 35427e1bb15c4054a01441bf811a0e08
I20260811 22:12:27.320278 21777 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: loading cluster ID for follower catalog manager: success
I20260811 22:12:27.324200 21777 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: acquiring CA information for follower catalog manager: success
I20260811 22:12:27.326886 21777 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:12:27.492317 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.485461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37661 (local address 127.0.0.1:41850)
0811 22:12:27.485727 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:27.485751 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:27.485946 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:27.486886 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:27.486891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:27.486921 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:27.487122 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:27.487132 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.488703 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.488709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:27.489925 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:27.489938 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.491420 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.491427 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:27.491661 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.491700 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:12:27.491813 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:12:27.491985 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 22:12:27.492362 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.485622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.10.173.188:37661)
0811 22:12:27.486566 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:27.486571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:27.486594 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:12:27.486655 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:27.486659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:27.486724 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:27.486859 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:27.487293 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.488537 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.490098 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.491169 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.491420 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.491620 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:27.491841 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 22:12:27.491983 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":628,"threads_started":1}
I20260811 22:12:27.493664 21721 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 22:12:27.494663 21762 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11e46714a91e4356a74769c0ceaaed9c. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:27.495009 21762 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: This master's current role is: LEARNER
W20260811 22:12:27.533212 21666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:27.533490 21666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:27.533591 21666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:27.541314 21666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:27.541445 21666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:27.541586 21666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:12:27.553534 21666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab
--principal=oryx/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:45861
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=39649
--tserver_master_addrs=127.10.173.190:32997,127.10.173.189:39265,127.10.173.188:37661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:27.554976 21666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:27.556423 21666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:27.573060 21790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:27.573256 21799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:27.573604 21785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:27.574038 21666 server_base.cc:1034] running on GCE node
I20260811 22:12:27.574707 21666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:27.575690 21666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:27.576923 21666 hybrid_clock.cc:648] HybridClock initialized: now 1786486347576842 us; error 35 us; skew 500 ppm
I20260811 22:12:27.577309 21666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:27 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486347, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:27.580881 21666 init.cc:377] Logged in from keytab as oryx/127.10.173.129@KRBTEST.COM (short username oryx)
I20260811 22:12:27.582778 21666 webserver.cc:460] Webserver started at http://127.10.173.129:39649/ using document root <none> and password file <none>
I20260811 22:12:27.583534 21666 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:27.583690 21666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:27.591053 21666 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 22:12:27.594762 21810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:27.596212 21666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:27.596491 21666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "f2a43230c04d4b64bd8fd1f05e505914"
format_stamp: "Formatted at 2026-08-11 22:12:23 on dist-test-slave-68xk"
server_key: "8b3367ad69b2c866eb2a7b024a8d4c60"
server_key_iv: "ecdffeb508e9e2ec42303e44d95ec033"
server_key_version: "encryptionkey@0"
I20260811 22:12:27.597601 21666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:27.611032 21666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:27.615512 21666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:27.615806 21666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:27.616945 21666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:27.618554 21666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:27.618702 21666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:27.618837 21666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:27.618971 21666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:27.663647 21666 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:45861
I20260811 22:12:27.663756 21990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:45861 every 8 connection(s)
I20260811 22:12:27.665279 21666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:27.670612 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 21666
I20260811 22:12:27.671020 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 20748
Aug 11 22:12:27 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486347, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.188@KRBTEST.COM
Aug 11 22:12:27 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486347, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.189@KRBTEST.COM
Aug 11 22:12:27 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486347, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
Aug 11 22:12:27 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.130@KRBTEST.COM
I20260811 22:12:27.706199 21779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.680819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:40067 (local address 127.10.173.188:37661)
0811 22:12:27.681128 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:27.681137 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:27.683608 (+ 2471us) server_negotiation.cc:407] Connection header received
0811 22:12:27.684335 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:27.684340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:27.684421 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:27.684498 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:27.687566 (+ 3068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.688728 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.690887 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.691592 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.691699 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.697183 (+ 5484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:27.697190 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:27.697195 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:27.697238 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:27.701792 (+ 4554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:27.702965 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:27.702972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:27.702975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:27.703168 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:27.703737 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:27.703745 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:27.703750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:27.704151 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:27.704308 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:27.705504 (+ 1196us) server_negotiation.cc:299] Negotiation successful
0811 22:12:27.705805 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260811 22:12:27.706462 22000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.668169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:47225 (local address 127.10.173.189:39265)
0811 22:12:27.674629 (+ 6460us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:27.674639 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:27.680174 (+ 5535us) server_negotiation.cc:407] Connection header received
0811 22:12:27.684466 (+ 4292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:27.684482 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:27.684574 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:27.684661 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:27.686921 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.688042 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.690355 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.691121 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.691245 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.698314 (+ 7069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:27.698323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:27.698329 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:27.698380 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:27.702891 (+ 4511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:27.703808 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:27.703814 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:27.703820 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:27.703955 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:27.704469 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:27.704474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:27.704477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:27.704912 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:27.705068 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:27.705782 (+ 714us) server_negotiation.cc:299] Negotiation successful
0811 22:12:27.706000 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6291,"thread_start_us":6151,"threads_started":1}
I20260811 22:12:27.707973 22009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.681296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37661 (local address 127.10.173.129:40067)
0811 22:12:27.683479 (+ 2183us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:27.683499 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:27.684157 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:27.684948 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:27.684961 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:27.686104 (+ 1143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:27.687365 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:27.687382 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.688929 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.688939 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:27.690714 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:27.690729 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.692031 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.692038 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:27.692452 (+ 414us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.693432 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:27.693471 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:27.696909 (+ 3438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:27.701973 (+ 5064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:27.701985 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:27.702004 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:27.702812 (+ 808us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:27.703340 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:27.703344 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:27.703348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:27.703592 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:27.704402 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:27.704413 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:27.704935 (+ 522us) client_negotiation.cc:769] Sending connection context
0811 22:12:27.706530 (+ 1595us) client_negotiation.cc:240] Negotiation successful
0811 22:12:27.706824 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2059,"mutex_wait_us":84,"thread_start_us":2013,"threads_started":1}
I20260811 22:12:27.707998 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.669096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.10.173.129:47225)
0811 22:12:27.679946 (+ 10850us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:27.680010 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:27.683541 (+ 3531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:27.684947 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:27.684961 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:27.685978 (+ 1017us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:27.686691 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:27.686711 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.688262 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.688275 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:27.690185 (+ 1910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:27.690202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.691954 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.691960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:27.692461 (+ 501us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.694026 (+ 1565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:27.694055 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:27.698108 (+ 4053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:27.703099 (+ 4991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:27.703108 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:27.703112 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:27.703678 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:27.704106 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:27.704110 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:27.704114 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:27.704337 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:27.705158 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:27.705163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:27.705442 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 22:12:27.706707 (+ 1265us) client_negotiation.cc:240] Negotiation successful
0811 22:12:27.706963 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10422,"mutex_wait_us":937,"thread_start_us":9965,"threads_started":1}
I20260811 22:12:27.708523 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.668263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:40451 (local address 127.10.173.190:32997)
0811 22:12:27.684191 (+ 15928us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:27.684199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:27.684219 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:12:27.684289 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:27.684295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:27.684374 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:27.684492 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:27.689409 (+ 4917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.690544 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.692645 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.693403 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.693547 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.698840 (+ 5293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:27.698850 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:27.698856 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:27.698903 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:27.702627 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:27.703579 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:27.703584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:27.703587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:27.703801 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:27.704363 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:27.704367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:27.704371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:27.704837 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:27.704994 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:27.707860 (+ 2866us) server_negotiation.cc:299] Negotiation successful
0811 22:12:27.708108 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15806,"thread_start_us":15697,"threads_started":1}
I20260811 22:12:27.708873 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.672402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.129:40451)
0811 22:12:27.683490 (+ 11088us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:27.683517 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:27.683696 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:27.684948 (+ 1252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:27.684961 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:27.688741 (+ 3780us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:27.689226 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:27.689239 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.690710 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.690716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:27.692480 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:27.692495 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.693962 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.693974 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:27.694245 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.695401 (+ 1156us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:27.695424 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:27.698511 (+ 3087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:27.702797 (+ 4286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:27.702803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:27.702807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:27.703421 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:27.703954 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:27.703960 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:27.703965 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:27.704218 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:27.705093 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:27.705100 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:27.705397 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 22:12:27.705666 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 22:12:27.706217 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10756,"mutex_wait_us":7,"spinlock_wait_cycles":1408,"thread_start_us":10552,"threads_started":1}
I20260811 22:12:27.709913 21993 heartbeater.cc:347] Connected to a master server at 127.10.173.188:37661
I20260811 22:12:27.709973 21994 heartbeater.cc:347] Connected to a master server at 127.10.173.190:32997
I20260811 22:12:27.710000 21996 heartbeater.cc:347] Connected to a master server at 127.10.173.189:39265
I20260811 22:12:27.710323 21994 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:27.710317 21996 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:27.710317 21993 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:27.711542 21994 heartbeater.cc:511] Master 127.10.173.190:32997 requested a full tablet report, sending...
I20260811 22:12:27.711602 21996 heartbeater.cc:511] Master 127.10.173.189:39265 requested a full tablet report, sending...
I20260811 22:12:27.711892 21993 heartbeater.cc:511] Master 127.10.173.188:37661 requested a full tablet report, sending...
I20260811 22:12:27.713845 20448 ts_manager.cc:194] Re-registered known tserver with Master: f2a43230c04d4b64bd8fd1f05e505914 (127.10.173.129:45861)
I20260811 22:12:27.714455 21393 ts_manager.cc:194] Registered new tserver with Master: f2a43230c04d4b64bd8fd1f05e505914 (127.10.173.129:45861)
I20260811 22:12:27.714679 21704 ts_manager.cc:194] Registered new tserver with Master: f2a43230c04d4b64bd8fd1f05e505914 (127.10.173.129:45861)
I20260811 22:12:27.715065 20448 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:40451
I20260811 22:12:27.716621 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.672122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42034 (local address 127.10.173.130:36603)
0811 22:12:27.678990 (+ 6868us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:27.679002 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:27.684524 (+ 5522us) server_negotiation.cc:407] Connection header received
0811 22:12:27.684845 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:27.684855 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:27.685137 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:27.685306 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:27.689456 (+ 4150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.691156 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.693535 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:27.694520 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:27.694623 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:27.702662 (+ 8039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:27.702681 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:27.702703 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:27.702754 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:27.707906 (+ 5152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:27.710306 (+ 2400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:27.710318 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:27.710322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:27.710669 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:27.711123 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:27.711129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:27.711132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:27.711846 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:27.715684 (+ 3838us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:27.715813 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 22:12:27.716104 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6639,"thread_start_us":6448,"threads_started":1}
I20260811 22:12:27.717923 20935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:27.806061 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 20748
I20260811 22:12:27.834584 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:36603
--local_ip_for_outbound_sockets=127.10.173.130
--tserver_master_addrs=127.10.173.190:32997
--webserver_port=44947
--webserver_interface=127.10.173.130
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab
--principal=oryx/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.173.190:32997,127.10.173.189:39265
--tserver_master_addrs=127.10.173.190:32997,127.10.173.189:39265,127.10.173.188:37661 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:27.989957 21633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: NON_VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: true } } }
I20260811 22:12:27.990473 21633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:27.995884 22016 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c: attempting to promote NON_VOTER ff65fc83f0854276a28f78343c6d7d31 to VOTER
I20260811 22:12:27.996555 22016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } }
I20260811 22:12:27.998664 21721 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 LEARNER]: Refusing update from remote peer 11e46714a91e4356a74769c0ceaaed9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 22:12:27.998767 21415 raft_consensus.cc:1261] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Refusing update from remote peer 11e46714a91e4356a74769c0ceaaed9c: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 22:12:27.999374 22016 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 22:12:27.999855 22014 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 22:12:28.002784 21633 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ff65fc83f0854276a28f78343c6d7d31 (127.10.173.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:28.003348 21415 raft_consensus.cc:2932] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ff65fc83f0854276a28f78343c6d7d31 (127.10.173.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:28.007062 21633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:28.007542 21633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:28.007887 22014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:28.008204 22014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:28.009876 21633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:28.010301 21633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:28.010948 21721 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, ff65fc83f0854276a28f78343c6d7d31 (127.10.173.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:28.017037 22021 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:28.017419 22021 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:28.018787 22020 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11e46714a91e4356a74769c0ceaaed9c" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:28.019191 22020 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: This master's current role is: FOLLOWER
W20260811 22:12:28.193786 22012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:28.194073 22012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:28.194226 22012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:28.201840 22012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:28.201983 22012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:28.202101 22012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:12:28.215605 22012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab
--principal=oryx/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:36603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=44947
--tserver_master_addrs=127.10.173.190:32997,127.10.173.189:39265,127.10.173.188:37661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:28.217239 22012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:28.218963 22012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:28.233976 22032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:28.234184 22033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:28.234974 22035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:28.235460 22012 server_base.cc:1034] running on GCE node
I20260811 22:12:28.236322 22012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:28.237295 22012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:28.238551 22012 hybrid_clock.cc:648] HybridClock initialized: now 1786486348238502 us; error 27 us; skew 500 ppm
I20260811 22:12:28.238880 22012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:28 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486348, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:28.242017 22012 init.cc:377] Logged in from keytab as oryx/127.10.173.130@KRBTEST.COM (short username oryx)
I20260811 22:12:28.243585 22012 webserver.cc:460] Webserver started at http://127.10.173.130:44947/ using document root <none> and password file <none>
I20260811 22:12:28.244303 22012 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:28.244457 22012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:28.250315 22012 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:12:28.253269 22047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:28.254875 22012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:28.255072 22012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "c94f10d8f018483fbc787ed2b498e390"
format_stamp: "Formatted at 2026-08-11 22:12:24 on dist-test-slave-68xk"
server_key: "6f7f21e4ef14093824e0f499c2a85f7a"
server_key_iv: "e824b27a01d5828e5e5fdd5f71f5669a"
server_key_version: "encryptionkey@0"
I20260811 22:12:28.256088 22012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:28.295720 22012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:28.300426 22012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:28.300772 22012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:28.302111 22012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:28.303700 22012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:28.303793 22012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:28.303906 22012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:28.304013 22012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:28.340394 22012 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:36603
I20260811 22:12:28.340487 22246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:36603 every 8 connection(s)
I20260811 22:12:28.341930 22012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:28.352052 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 22012
I20260811 22:12:28.352394 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 20956
Aug 11 22:12:28 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486348, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.189@KRBTEST.COM
I20260811 22:12:28.378897 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:28.344940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:43167 (local address 127.10.173.189:39265)
0811 22:12:28.345607 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:28.345618 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:28.347080 (+ 1462us) server_negotiation.cc:407] Connection header received
0811 22:12:28.351212 (+ 4132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:28.351224 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:28.351343 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:28.351451 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:28.354879 (+ 3428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.356460 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.359971 (+ 3511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.360867 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.360998 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:28.367936 (+ 6938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:28.367947 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:28.367955 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:28.368025 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:28.373063 (+ 5038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:28.374628 (+ 1565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:28.374638 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:28.374645 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:28.374859 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:28.375625 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:28.375634 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:28.375640 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:28.376190 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:28.376386 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:28.377874 (+ 1488us) server_negotiation.cc:299] Negotiation successful
0811 22:12:28.378212 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"spinlock_wait_cycles":7552,"thread_start_us":428,"threads_started":1}
I20260811 22:12:28.382679 22253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:28.345647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.10.173.130:43167)
0811 22:12:28.346858 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:28.346935 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:28.350779 (+ 3844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:28.352318 (+ 1539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:28.352336 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:28.353469 (+ 1133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:28.354578 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:28.354611 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.356788 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.356797 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:28.359737 (+ 2940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:28.359759 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.361508 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.361518 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:28.361965 (+ 447us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:28.363086 (+ 1121us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:28.363137 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:28.367546 (+ 4409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:28.373467 (+ 5921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:28.373481 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:28.373509 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:28.374396 (+ 887us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:28.375057 (+ 661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:28.375065 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:28.375071 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:28.375436 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:28.376512 (+ 1076us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:28.376524 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:28.377279 (+ 755us) client_negotiation.cc:769] Sending connection context
0811 22:12:28.379641 (+ 2362us) client_negotiation.cc:240] Negotiation successful
0811 22:12:28.380479 (+ 838us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":816,"threads_started":1}
I20260811 22:12:28.384975 22249 heartbeater.cc:347] Connected to a master server at 127.10.173.189:39265
I20260811 22:12:28.385397 22249 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:28.386693 22249 heartbeater.cc:511] Master 127.10.173.189:39265 requested a full tablet report, sending...
I20260811 22:12:28.389096 21393 ts_manager.cc:194] Registered new tserver with Master: c94f10d8f018483fbc787ed2b498e390 (127.10.173.130:36603)
Aug 11 22:12:28 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486348, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.188@KRBTEST.COM
Aug 11 22:12:28 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486348, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
Aug 11 22:12:28 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.131@KRBTEST.COM
I20260811 22:12:28.405079 22258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:28.348488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:34959 (local address 127.10.173.188:37661)
0811 22:12:28.378844 (+ 30356us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:28.378853 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:28.380718 (+ 1865us) server_negotiation.cc:407] Connection header received
0811 22:12:28.380811 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:28.380818 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:28.380945 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:28.381057 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:28.383166 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.384440 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.391440 (+ 7000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.392133 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.392273 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:28.397343 (+ 5070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:28.397352 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:28.397359 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:28.397406 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:28.401180 (+ 3774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:28.402345 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:28.402353 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:28.402359 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:28.402608 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:28.403146 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:28.403154 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:28.403160 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:28.403542 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:28.403696 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:28.404335 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 22:12:28.404619 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30160,"thread_start_us":31364,"threads_started":1}
I20260811 22:12:28.405079 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:28.347848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37661 (local address 127.10.173.130:34959)
0811 22:12:28.380437 (+ 32589us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:28.380463 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:28.380588 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:28.381359 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:28.381367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:28.382238 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:28.382962 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:28.382979 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.384627 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.384633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:28.391268 (+ 6635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:28.391284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.392333 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.392339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:28.392521 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:28.394511 (+ 1990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:28.394534 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:28.397106 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:28.401429 (+ 4323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:28.401435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:28.401438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:28.402093 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:28.402823 (+ 730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:28.402831 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:28.402836 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:28.402995 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:28.403688 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:28.403693 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:28.403957 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 22:12:28.404132 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 22:12:28.404456 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32441,"thread_start_us":31780,"threads_started":1}
I20260811 22:12:28.406735 22247 heartbeater.cc:347] Connected to a master server at 127.10.173.188:37661
I20260811 22:12:28.406905 22247 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:28.407318 22247 heartbeater.cc:511] Master 127.10.173.188:37661 requested a full tablet report, sending...
I20260811 22:12:28.408464 21704 ts_manager.cc:194] Registered new tserver with Master: c94f10d8f018483fbc787ed2b498e390 (127.10.173.130:36603)
I20260811 22:12:28.410175 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:28.346352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.130:32823)
0811 22:12:28.379373 (+ 33021us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:28.379397 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:28.379546 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:28.390256 (+ 10710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:28.390264 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:28.391302 (+ 1038us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:28.391927 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:28.391939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.393248 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.393255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:28.394233 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:28.394245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.395369 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.395377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:28.395591 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:28.396310 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:28.396333 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:28.400361 (+ 4028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:28.405528 (+ 5167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:28.405535 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:28.405541 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:28.406071 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:28.407688 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:28.407692 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:28.407695 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:28.407931 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:28.409043 (+ 1112us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:28.409048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:28.409210 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 22:12:28.409443 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 22:12:28.409736 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32858,"thread_start_us":36255,"threads_started":1}
I20260811 22:12:28.410270 22252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:28.345280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:32823 (local address 127.10.173.190:32997)
0811 22:12:28.389775 (+ 44495us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:28.389784 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:28.389812 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:12:28.389890 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:28.389898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:28.390012 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:28.390839 (+ 827us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:28.392085 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.393120 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.394380 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.395110 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.395251 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:28.400619 (+ 5368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:28.400629 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:28.400636 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:28.400692 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:28.405380 (+ 4688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:28.406448 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:28.406457 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:28.406463 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:28.406603 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:28.408087 (+ 1484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:28.408094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:28.408098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:28.408455 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:28.408648 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:28.409584 (+ 936us) server_negotiation.cc:299] Negotiation successful
0811 22:12:28.409835 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44318,"thread_start_us":44174,"threads_started":1}
I20260811 22:12:28.411340 22248 heartbeater.cc:347] Connected to a master server at 127.10.173.190:32997
I20260811 22:12:28.411490 22248 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:28.411896 22264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:28.353940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39020 (local address 127.10.173.131:33837)
0811 22:12:28.384638 (+ 30698us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:28.384651 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:28.385904 (+ 1253us) server_negotiation.cc:407] Connection header received
0811 22:12:28.386006 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:28.386018 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:28.386353 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:28.386590 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:28.391602 (+ 5012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.393180 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.394891 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:28.396338 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:28.396512 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:28.403613 (+ 7101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:28.403633 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:28.403653 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:28.403695 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:28.407691 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:28.408640 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:28.408650 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:28.408653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:28.408857 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:28.409295 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:28.409301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:28.409305 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:28.409754 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:28.410104 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:28.410975 (+ 871us) server_negotiation.cc:299] Negotiation successful
0811 22:12:28.411290 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29603,"thread_start_us":29756,"threads_started":1}
I20260811 22:12:28.412117 22248 heartbeater.cc:511] Master 127.10.173.190:32997 requested a full tablet report, sending...
I20260811 22:12:28.413264 20448 ts_manager.cc:194] Re-registered known tserver with Master: c94f10d8f018483fbc787ed2b498e390 (127.10.173.130:36603)
I20260811 22:12:28.413787 21144 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:28.414326 20448 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:32823
I20260811 22:12:28.505769 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 20956
I20260811 22:12:28.532557 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:33837
--local_ip_for_outbound_sockets=127.10.173.131
--tserver_master_addrs=127.10.173.190:32997
--webserver_port=36667
--webserver_interface=127.10.173.131
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab
--principal=oryx/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.173.190:32997,127.10.173.189:39265
--tserver_master_addrs=127.10.173.190:32997,127.10.173.189:39265,127.10.173.188:37661 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:28.717832 21994 heartbeater.cc:503] Master 127.10.173.190:32997 was elected leader, sending a full tablet report...
W20260811 22:12:28.886263 22267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:28.886693 22267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:28.886857 22267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:28.894986 22267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:28.895131 22267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:28.895251 22267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:12:28.905921 22267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab
--principal=oryx/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:33837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=36667
--tserver_master_addrs=127.10.173.190:32997,127.10.173.189:39265,127.10.173.188:37661
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:28.907455 22267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:28.908728 22267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:28.923132 22280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:28.923179 22276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:28.923178 22275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:28.923873 22267 server_base.cc:1034] running on GCE node
I20260811 22:12:28.924633 22267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:28.925598 22267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:28.926831 22267 hybrid_clock.cc:648] HybridClock initialized: now 1786486348926773 us; error 52 us; skew 500 ppm
I20260811 22:12:28.927165 22267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:28 dist-test-slave-68xk krb5kdc[20378](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486348, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:28.930490 22267 init.cc:377] Logged in from keytab as oryx/127.10.173.131@KRBTEST.COM (short username oryx)
I20260811 22:12:28.932102 22267 webserver.cc:460] Webserver started at http://127.10.173.131:36667/ using document root <none> and password file <none>
I20260811 22:12:28.932785 22267 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:28.932906 22267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:28.938815 22267 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260811 22:12:28.941828 22290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:28.943292 22267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:28.943445 22267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "3d7de0600ef3442180518f144d68006b"
format_stamp: "Formatted at 2026-08-11 22:12:24 on dist-test-slave-68xk"
server_key: "6abcf3972b221b756e3fd401b96f7987"
server_key_iv: "3f77641b5f44cc5539ce7ccf70285316"
server_key_version: "encryptionkey@0"
I20260811 22:12:28.944532 22267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:28.957106 22267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:28.961617 22267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:28.961951 22267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:28.963033 22267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:28.964540 22267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:28.964638 22267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:28.964782 22267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:28.964893 22267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:29.007252 22267 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:33837
I20260811 22:12:29.007865 22478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:33837 every 8 connection(s)
I20260811 22:12:29.009475 22267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:29.019522 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 22267
Aug 11 22:12:29 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486348, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
Aug 11 22:12:29 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486348, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.188@KRBTEST.COM
I20260811 22:12:29.046593 22486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.012704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:51415 (local address 127.10.173.190:32997)
0811 22:12:29.013296 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.013303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.017977 (+ 4674us) server_negotiation.cc:407] Connection header received
0811 22:12:29.018331 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.018337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.018472 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.018600 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:29.021541 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.023261 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.026574 (+ 3313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.027559 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.027734 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.037380 (+ 9646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:29.037390 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:29.037394 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:29.037441 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:29.041407 (+ 3966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.042630 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.042637 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.042640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.042813 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.043547 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.043557 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.043563 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.043984 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:29.044146 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.045571 (+ 1425us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.045889 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":316,"threads_started":1}
I20260811 22:12:29.049357 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.016241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:47237 (local address 127.10.173.188:37661)
0811 22:12:29.016747 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.016754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.016778 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:12:29.025199 (+ 8421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.025204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.025294 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.025426 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:29.028377 (+ 2951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.029917 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.032128 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.032583 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.032749 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.039461 (+ 6712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:29.039471 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:29.039479 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:29.039529 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:29.043414 (+ 3885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.044366 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.044371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.044374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.044535 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.045009 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.045014 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.045018 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.045379 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:29.045520 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.047092 (+ 1572us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.047366 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":256,"threads_started":1}
I20260811 22:12:29.049556 22487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.013374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37661 (local address 127.10.173.131:47237)
0811 22:12:29.014455 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.014519 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:29.017999 (+ 3480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:29.025451 (+ 7452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:29.025456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:29.026168 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:29.028201 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.028223 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.030108 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.030116 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.031959 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.031972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.032911 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.032923 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.033254 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.034619 (+ 1365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:29.034653 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:29.039249 (+ 4596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:29.043564 (+ 4315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:29.043569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:29.043572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:29.044132 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:29.044671 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:29.044676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:29.044679 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:29.044872 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:29.045582 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:29.045587 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:29.045740 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 22:12:29.045890 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 22:12:29.046495 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":833,"threads_started":1}
I20260811 22:12:29.049865 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.014107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.131:51415)
0811 22:12:29.017809 (+ 3702us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.017840 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:29.018012 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:29.018947 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:29.018963 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:29.020240 (+ 1277us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:29.021283 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.021313 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.023443 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.023452 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.026372 (+ 2920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.026394 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.027864 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.027874 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.028701 (+ 827us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.029999 (+ 1298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:29.030040 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:29.037026 (+ 6986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:29.041608 (+ 4582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:29.041619 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:29.041637 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:29.042468 (+ 831us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:29.042998 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:29.043007 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:29.043014 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:29.043391 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:29.044308 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:29.044321 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:29.044933 (+ 612us) client_negotiation.cc:769] Sending connection context
0811 22:12:29.046979 (+ 2046us) client_negotiation.cc:240] Negotiation successful
0811 22:12:29.047408 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3521,"mutex_wait_us":480,"thread_start_us":2616,"threads_started":1}
Aug 11 22:12:29 dist-test-slave-68xk krb5kdc[20378](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486348, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.189@KRBTEST.COM
I20260811 22:12:29.052002 22483 heartbeater.cc:347] Connected to a master server at 127.10.173.190:32997
I20260811 22:12:29.052158 22479 heartbeater.cc:347] Connected to a master server at 127.10.173.188:37661
I20260811 22:12:29.052385 22483 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:29.052402 22479 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:29.053967 22479 heartbeater.cc:511] Master 127.10.173.188:37661 requested a full tablet report, sending...
I20260811 22:12:29.055614 22483 heartbeater.cc:511] Master 127.10.173.190:32997 requested a full tablet report, sending...
I20260811 22:12:29.056985 20448 ts_manager.cc:194] Re-registered known tserver with Master: 3d7de0600ef3442180518f144d68006b (127.10.173.131:33837)
I20260811 22:12:29.057691 21704 ts_manager.cc:194] Registered new tserver with Master: 3d7de0600ef3442180518f144d68006b (127.10.173.131:33837)
I20260811 22:12:29.058202 20448 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:51415
I20260811 22:12:29.061796 22494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.018719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:45675 (local address 127.10.173.189:39265)
0811 22:12:29.019304 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.019314 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.038463 (+ 19149us) server_negotiation.cc:407] Connection header received
0811 22:12:29.040696 (+ 2233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.040706 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.040805 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.040929 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:29.042391 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.043560 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.045824 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.046558 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.046652 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.053101 (+ 6449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:29.053108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:29.053112 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:29.053155 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:29.057241 (+ 4086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.059256 (+ 2015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.059263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.059270 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.059430 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.059964 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.059972 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.059977 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.060418 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:29.060551 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.061078 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.061344 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":2175,"threads_started":1}
I20260811 22:12:29.062529 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.018592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.10.173.131:45675)
0811 22:12:29.038317 (+ 19725us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.038348 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:29.040495 (+ 2147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:29.041122 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:29.041127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:29.041763 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:29.042195 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.042209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.043773 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.043782 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.045651 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.045673 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.047179 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.047185 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.047385 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.048166 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:29.048188 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:29.052862 (+ 4674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:29.058488 (+ 5626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:29.058493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:29.058497 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:29.059120 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:29.059582 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:29.059589 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:29.059593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:29.059806 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:29.060656 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:29.060660 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:29.060816 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 22:12:29.060964 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 22:12:29.061344 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19560,"thread_start_us":19389,"threads_started":1}
I20260811 22:12:29.063665 22484 heartbeater.cc:347] Connected to a master server at 127.10.173.189:39265
I20260811 22:12:29.063874 22484 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:29.064339 22484 heartbeater.cc:511] Master 127.10.173.189:39265 requested a full tablet report, sending...
I20260811 22:12:29.065989 21393 ts_manager.cc:194] Registered new tserver with Master: 3d7de0600ef3442180518f144d68006b (127.10.173.131:33837)
I20260811 22:12:29.079525 22505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.043839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41934 (local address 127.10.173.189:39265)
0811 22:12:29.048008 (+ 4169us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.048015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.051799 (+ 3784us) server_negotiation.cc:407] Connection header received
0811 22:12:29.053301 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.053311 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.053687 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.053804 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:29.062225 (+ 8421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.063716 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.066529 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.067137 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.067308 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.071595 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:29.071604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:29.071611 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:29.071661 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:29.076040 (+ 4379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.077000 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.077005 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.077007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.077203 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.077790 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.077796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.077799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.078230 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:29.078552 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.078900 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.079167 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4051,"thread_start_us":3890,"threads_started":1}
I20260811 22:12:29.084427 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.050121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.10.173.188:37661)
0811 22:12:29.050847 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.050855 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.059326 (+ 8471us) server_negotiation.cc:407] Connection header received
0811 22:12:29.059988 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.059995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.060119 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.060198 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:29.063467 (+ 3269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.064356 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.069710 (+ 5354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.070782 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.070948 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.075518 (+ 4570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:29.075528 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:29.075534 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:29.075581 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:29.080709 (+ 5128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.081590 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.081596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.081600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.081820 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.082276 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.082280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.082282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.082803 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:29.083083 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.083749 (+ 666us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.084008 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591}
I20260811 22:12:29.084481 22486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.053567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36518 (local address 127.10.173.190:32997)
0811 22:12:29.056199 (+ 2632us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.056209 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.062503 (+ 6294us) server_negotiation.cc:407] Connection header received
0811 22:12:29.062860 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.062866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.062952 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.063040 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:29.069334 (+ 6294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.070292 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.072562 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.073134 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.073306 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.076369 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:29.076378 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:29.076385 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:29.076440 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:29.080892 (+ 4452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.081740 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.081747 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.081751 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.081906 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.082440 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.082449 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.082455 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.082841 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:29.083085 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.083825 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.084060 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2457}
I20260811 22:12:29.088820 20448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:29.096398 20448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:29.143410 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.121153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.10.173.131:33837)
0811 22:12:29.132867 (+ 11714us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.132874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.132908 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:12:29.132993 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.132998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.133316 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.133518 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:29.134360 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.136216 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.139958 (+ 3742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.141729 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.141917 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.142132 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.142782 (+ 650us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.142953 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11551,"thread_start_us":12595,"threads_started":1}
I20260811 22:12:29.143386 22529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.120792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:33837 (local address 127.0.0.1:39026)
0811 22:12:29.131833 (+ 11041us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.131857 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:29.131999 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:29.133512 (+ 1513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:29.133519 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:29.133555 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:29.134156 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.134174 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.136602 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.136607 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.139755 (+ 3148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.139773 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.142194 (+ 2421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.142200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.142567 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.142608 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 22:12:29.142718 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 22:12:29.142919 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10879,"thread_start_us":10227,"threads_started":1}
I20260811 22:12:29.144758 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.120973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42050 (local address 127.10.173.130:36603)
0811 22:12:29.136360 (+ 15387us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.136367 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.136390 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:12:29.136471 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.136475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.136629 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.136827 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:29.138207 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.139290 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.142218 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.143712 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.143883 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.144060 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.144291 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.144463 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15208,"thread_start_us":15083,"threads_started":1}
I20260811 22:12:29.144793 22526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.120308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:36603 (local address 127.0.0.1:42050)
0811 22:12:29.131355 (+ 11047us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.131371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:29.131481 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:29.137287 (+ 5806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:29.137291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:29.137315 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:29.138035 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.138049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.139452 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.139458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.141800 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.141817 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.143888 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.143895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.144095 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.144130 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:12:29.144246 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 22:12:29.144439 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10906,"thread_start_us":11952,"threads_started":1}
I20260811 22:12:29.145346 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.121514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45861 (local address 127.0.0.1:58814)
0811 22:12:29.133136 (+ 11622us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.133154 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:29.133284 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:29.135889 (+ 2605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:29.135897 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:29.135938 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:29.136395 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.136406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.138763 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.138770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.141209 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.141225 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.144128 (+ 2903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.144135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.144324 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.144362 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:12:29.144459 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 22:12:29.144602 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10897,"mutex_wait_us":27,"thread_start_us":10996,"threads_started":1}
I20260811 22:12:29.145543 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.121899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58814 (local address 127.10.173.129:45861)
0811 22:12:29.135237 (+ 13338us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.135246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.135272 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:12:29.135368 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.135376 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.135594 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.135800 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:29.136595 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.138579 (+ 1984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.141442 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.143850 (+ 2408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.144061 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.144250 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.144533 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.144700 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13165,"thread_start_us":12991,"threads_started":1}
I20260811 22:12:29.149242 22139 tablet_service.cc:1467] Processing CreateTablet for tablet 25b9c12d95e04bb399db6f26b17b99f1 (DEFAULT_TABLE table=test-table [id=ea61f650b8fa44919877dda75d0d77f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:29.149190 22374 tablet_service.cc:1467] Processing CreateTablet for tablet 25b9c12d95e04bb399db6f26b17b99f1 (DEFAULT_TABLE table=test-table [id=ea61f650b8fa44919877dda75d0d77f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:29.150592 22139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b9c12d95e04bb399db6f26b17b99f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:29.150712 22374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b9c12d95e04bb399db6f26b17b99f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:29.155934 21886 tablet_service.cc:1467] Processing CreateTablet for tablet 25b9c12d95e04bb399db6f26b17b99f1 (DEFAULT_TABLE table=test-table [id=ea61f650b8fa44919877dda75d0d77f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:29.157629 21886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25b9c12d95e04bb399db6f26b17b99f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:29.163899 22531 tablet_bootstrap.cc:492] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: Bootstrap starting.
I20260811 22:12:29.167249 22533 tablet_bootstrap.cc:492] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: Bootstrap starting.
I20260811 22:12:29.167992 22531 tablet_bootstrap.cc:654] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:29.170224 22531 log.cc:824] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:29.171536 22533 tablet_bootstrap.cc:654] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:29.172894 22533 log.cc:824] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:29.173434 22536 tablet_bootstrap.cc:492] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: Bootstrap starting.
I20260811 22:12:29.174333 22531 tablet_bootstrap.cc:492] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: No bootstrap required, opened a new log
I20260811 22:12:29.174747 22531 ts_tablet_manager.cc:1397] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260811 22:12:29.177070 22536 tablet_bootstrap.cc:654] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:29.177506 22533 tablet_bootstrap.cc:492] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: No bootstrap required, opened a new log
I20260811 22:12:29.177781 22533 ts_tablet_manager.cc:1397] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260811 22:12:29.178346 22536 log.cc:824] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:29.181430 22536 tablet_bootstrap.cc:492] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: No bootstrap required, opened a new log
I20260811 22:12:29.181721 22536 ts_tablet_manager.cc:1397] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 22:12:29.186662 22531 raft_consensus.cc:348] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } }
I20260811 22:12:29.187153 22531 raft_consensus.cc:374] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:29.187254 22531 raft_consensus.cc:729] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c94f10d8f018483fbc787ed2b498e390, State: Initialized, Role: FOLLOWER
I20260811 22:12:29.187933 22531 consensus_queue.cc:260] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } }
I20260811 22:12:29.189077 22533 raft_consensus.cc:348] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } }
I20260811 22:12:29.189512 22533 raft_consensus.cc:374] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:29.189633 22533 raft_consensus.cc:729] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7de0600ef3442180518f144d68006b, State: Initialized, Role: FOLLOWER
I20260811 22:12:29.189992 22531 ts_tablet_manager.cc:1428] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 22:12:29.190282 22248 heartbeater.cc:503] Master 127.10.173.190:32997 was elected leader, sending a full tablet report...
I20260811 22:12:29.190346 22533 consensus_queue.cc:260] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } }
I20260811 22:12:29.190354 22536 raft_consensus.cc:348] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } }
I20260811 22:12:29.190799 22536 raft_consensus.cc:374] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:29.190897 22536 raft_consensus.cc:729] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2a43230c04d4b64bd8fd1f05e505914, State: Initialized, Role: FOLLOWER
I20260811 22:12:29.191543 22536 consensus_queue.cc:260] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } }
I20260811 22:12:29.192732 22533 ts_tablet_manager.cc:1428] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: Time spent starting tablet: real 0.015s user 0.007s sys 0.007s
I20260811 22:12:29.193682 22483 heartbeater.cc:503] Master 127.10.173.190:32997 was elected leader, sending a full tablet report...
I20260811 22:12:29.200356 22536 ts_tablet_manager.cc:1428] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: Time spent starting tablet: real 0.018s user 0.013s sys 0.003s
W20260811 22:12:29.289497 22491 tablet.cc:2369] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:29.314503 22541 raft_consensus.cc:482] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:29.314872 22541 raft_consensus.cc:504] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } }
I20260811 22:12:29.316763 22541 leader_election.cc:288] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d7de0600ef3442180518f144d68006b (127.10.173.131:33837), f2a43230c04d4b64bd8fd1f05e505914 (127.10.173.129:45861)
I20260811 22:12:29.325702 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.317078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:55107 (local address 127.10.173.131:33837)
0811 22:12:29.317335 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.317343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.317943 (+ 600us) server_negotiation.cc:407] Connection header received
0811 22:12:29.318178 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.318182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.318248 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.318362 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:29.319200 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.320351 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.323067 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.324421 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.324601 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.324686 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.325297 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.325434 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:12:29.325702 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.317306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:38843 (local address 127.10.173.129:45861)
0811 22:12:29.317550 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.317556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.318324 (+ 768us) server_negotiation.cc:407] Connection header received
0811 22:12:29.318543 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.318547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.318625 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.318728 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:29.319377 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.320400 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.323206 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.324548 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.324690 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.324762 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.325297 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.325434 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:12:29.326133 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.316956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:33837 (local address 127.10.173.130:55107)
0811 22:12:29.317802 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.317827 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:29.318009 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:29.318386 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:29.318391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:29.318496 (+ 105us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:29.319030 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.319040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.320569 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.320574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.322910 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.322924 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.324747 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.324756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.325008 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.325136 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 22:12:29.325241 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:12:29.325491 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"mutex_wait_us":160,"thread_start_us":610,"threads_started":1}
I20260811 22:12:29.326140 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.317075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45861 (local address 127.10.173.130:38843)
0811 22:12:29.318220 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.318234 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:29.318374 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:29.318743 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:29.318747 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:29.318775 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:29.319172 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.319181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.320561 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.320566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.323060 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:29.323079 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.324776 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.324781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:29.324992 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.325136 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 22:12:29.325242 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:12:29.325491 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":135,"thread_start_us":843,"threads_started":1}
I20260811 22:12:29.327432 21921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b9c12d95e04bb399db6f26b17b99f1" candidate_uuid: "c94f10d8f018483fbc787ed2b498e390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a43230c04d4b64bd8fd1f05e505914" is_pre_election: true
I20260811 22:12:29.327457 22403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b9c12d95e04bb399db6f26b17b99f1" candidate_uuid: "c94f10d8f018483fbc787ed2b498e390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d7de0600ef3442180518f144d68006b" is_pre_election: true
I20260811 22:12:29.328161 22403 raft_consensus.cc:2454] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c94f10d8f018483fbc787ed2b498e390 in term 0.
I20260811 22:12:29.328164 21921 raft_consensus.cc:2454] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c94f10d8f018483fbc787ed2b498e390 in term 0.
I20260811 22:12:29.328939 22054 leader_election.cc:302] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3d7de0600ef3442180518f144d68006b, c94f10d8f018483fbc787ed2b498e390, f2a43230c04d4b64bd8fd1f05e505914; no voters:
I20260811 22:12:29.329473 22541 raft_consensus.cc:2781] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:29.329747 22541 raft_consensus.cc:482] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:29.329939 22541 raft_consensus.cc:3037] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:29.332636 22541 raft_consensus.cc:504] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } }
I20260811 22:12:29.333207 22541 leader_election.cc:288] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [CANDIDATE]: Term 1 election: Requested vote from peers 3d7de0600ef3442180518f144d68006b (127.10.173.131:33837), f2a43230c04d4b64bd8fd1f05e505914 (127.10.173.129:45861)
I20260811 22:12:29.333774 21921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b9c12d95e04bb399db6f26b17b99f1" candidate_uuid: "c94f10d8f018483fbc787ed2b498e390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a43230c04d4b64bd8fd1f05e505914"
I20260811 22:12:29.333760 22403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25b9c12d95e04bb399db6f26b17b99f1" candidate_uuid: "c94f10d8f018483fbc787ed2b498e390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d7de0600ef3442180518f144d68006b"
I20260811 22:12:29.333976 21921 raft_consensus.cc:3037] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:29.333976 22403 raft_consensus.cc:3037] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:29.337353 22403 raft_consensus.cc:2454] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c94f10d8f018483fbc787ed2b498e390 in term 1.
I20260811 22:12:29.337353 21921 raft_consensus.cc:2454] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c94f10d8f018483fbc787ed2b498e390 in term 1.
I20260811 22:12:29.337793 22054 leader_election.cc:302] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d7de0600ef3442180518f144d68006b, c94f10d8f018483fbc787ed2b498e390; no voters:
I20260811 22:12:29.338145 22541 raft_consensus.cc:2781] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:29.338685 22541 raft_consensus.cc:686] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 1 LEADER]: Becoming Leader. State: Replica: c94f10d8f018483fbc787ed2b498e390, State: Running, Role: LEADER
I20260811 22:12:29.339257 22541 consensus_queue.cc:237] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } }
I20260811 22:12:29.351473 20448 catalog_manager.cc:5495] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 reported cstate change: term changed from 0 to 1, leader changed from <none> to c94f10d8f018483fbc787ed2b498e390 (127.10.173.130). New cstate: current_term: 1 leader_uuid: "c94f10d8f018483fbc787ed2b498e390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } health_report { overall_health: UNKNOWN } } }
I20260811 22:12:29.379660 22527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.373563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42062 (local address 127.10.173.130:36603)
0811 22:12:29.373837 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.373843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.374006 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:12:29.374260 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.374264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.374341 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.374557 (+ 216us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:29.375033 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.375845 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.377296 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.377846 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.377997 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.378530 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:29.379026 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:29.379190 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.379294 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.379377 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:12:29.388244 21921 raft_consensus.cc:1261] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914 [term 1 FOLLOWER]: Refusing update from remote peer c94f10d8f018483fbc787ed2b498e390: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:29.388253 22403 raft_consensus.cc:1261] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b [term 1 FOLLOWER]: Refusing update from remote peer c94f10d8f018483fbc787ed2b498e390: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:29.389310 22555 consensus_queue.cc:1035] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:29.389631 22541 consensus_queue.cc:1035] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:12:29.392939 22250 tablet.cc:2369] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:29.404422 22563 mvcc.cc:205] Tried to move back new op lower bound from 7317448087084142592 to 7317448086898896896. Current Snapshot: MvccSnapshot[applied={T|T < 7317448087084142592}]
I20260811 22:12:29.426887 20446 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:29.427258 20446 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:29.434049 20446 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 11e46714a91e4356a74769c0ceaaed9c: Sending DeleteTablet for 3 replicas of tablet 25b9c12d95e04bb399db6f26b17b99f1
I20260811 22:12:29.435900 22139 tablet_service.cc:1514] Processing DeleteTablet for tablet 25b9c12d95e04bb399db6f26b17b99f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:29 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:42050
I20260811 22:12:29.435940 22374 tablet_service.cc:1514] Processing DeleteTablet for tablet 25b9c12d95e04bb399db6f26b17b99f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:29 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:39026
I20260811 22:12:29.437212 21886 tablet_service.cc:1514] Processing DeleteTablet for tablet 25b9c12d95e04bb399db6f26b17b99f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:29 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:58814
I20260811 22:12:29.441893 22578 tablet_replica.cc:330] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: stopping tablet replica
I20260811 22:12:29.442461 22578 raft_consensus.cc:2229] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:29.443318 22578 raft_consensus.cc:2258] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:29.444232 22580 tablet_replica.cc:330] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: stopping tablet replica
I20260811 22:12:29.444654 22580 raft_consensus.cc:2229] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:29.445499 22580 raft_consensus.cc:2258] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:29.445771 22578 ts_tablet_manager.cc:1905] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:29.449956 22583 tablet_replica.cc:330] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: stopping tablet replica
I20260811 22:12:29.450026 22580 ts_tablet_manager.cc:1905] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:29.450663 22583 raft_consensus.cc:2229] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:29.451319 22583 raft_consensus.cc:2258] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:29.452839 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 20398
I20260811 22:12:29.453915 20493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:29.453925 22578 ts_tablet_manager.cc:1918] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:29.454104 22578 log.cc:1196] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/25b9c12d95e04bb399db6f26b17b99f1
I20260811 22:12:29.454794 22578 ts_tablet_manager.cc:1939] T 25b9c12d95e04bb399db6f26b17b99f1 P c94f10d8f018483fbc787ed2b498e390: Deleting consensus metadata
I20260811 22:12:29.455981 22583 ts_tablet_manager.cc:1905] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:29.466104 22583 ts_tablet_manager.cc:1918] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:29.466312 22583 log.cc:1196] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/25b9c12d95e04bb399db6f26b17b99f1
I20260811 22:12:29.467854 22580 ts_tablet_manager.cc:1918] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:29.467917 22583 ts_tablet_manager.cc:1939] T 25b9c12d95e04bb399db6f26b17b99f1 P f2a43230c04d4b64bd8fd1f05e505914: Deleting consensus metadata
I20260811 22:12:29.468051 22580 log.cc:1196] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/25b9c12d95e04bb399db6f26b17b99f1
I20260811 22:12:29.468631 22580 ts_tablet_manager.cc:1939] T 25b9c12d95e04bb399db6f26b17b99f1 P 3d7de0600ef3442180518f144d68006b: Deleting consensus metadata
I20260811 22:12:29.482371 22505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.440557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.10.173.189:39265)
0811 22:12:29.440819 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.440829 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.453350 (+ 12521us) server_negotiation.cc:407] Connection header received
0811 22:12:29.453550 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.453559 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.453660 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.453784 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:29.465325 (+ 11541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.466526 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.469340 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.470343 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.470501 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.474847 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:29.474854 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:29.474859 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:29.474901 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:29.479042 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.479903 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.479909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.479913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.480085 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.480572 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.480578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.480582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.480950 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:29.481207 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.481630 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.481915 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:12:29.501107 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.452207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41872 (local address 127.10.173.188:37661)
0811 22:12:29.452381 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.452387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.482373 (+ 29986us) server_negotiation.cc:407] Connection header received
0811 22:12:29.482730 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.482737 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.482835 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.482912 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:29.484943 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.486178 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.488156 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.488864 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.488976 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.493389 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:29.493396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:29.493401 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:29.493444 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:29.497259 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.498235 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.498240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.498244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.498491 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.499280 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.499285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.499289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.499659 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:29.499959 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.500355 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.500588 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:12:29.603255 20421 catalog_manager.cc:4841] TS f2a43230c04d4b64bd8fd1f05e505914 (127.10.173.129:45861): tablet 25b9c12d95e04bb399db6f26b17b99f1 (table test-table [id=ea61f650b8fa44919877dda75d0d77f4]) successfully deleted
I20260811 22:12:29.604805 20427 catalog_manager.cc:4841] TS 3d7de0600ef3442180518f144d68006b (127.10.173.131:33837): tablet 25b9c12d95e04bb399db6f26b17b99f1 (table test-table [id=ea61f650b8fa44919877dda75d0d77f4]) successfully deleted
I20260811 22:12:29.605341 20422 catalog_manager.cc:4841] TS c94f10d8f018483fbc787ed2b498e390 (127.10.173.130:36603): tablet 25b9c12d95e04bb399db6f26b17b99f1 (table test-table [id=ea61f650b8fa44919877dda75d0d77f4]) successfully deleted
I20260811 22:12:29.605671 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 20398
I20260811 22:12:29.657872 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.642118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41876 (local address 127.10.173.188:37661)
0811 22:12:29.642363 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.642370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.642392 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:12:29.642477 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.642482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.642551 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.642649 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:29.644296 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.645256 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.647273 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.647895 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.648009 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.651371 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:29.651376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:29.651379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:29.651413 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:29.654920 (+ 3507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.655727 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.655733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.655737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.655874 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.656247 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.656252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.656256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.656668 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:29.656924 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.657283 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.657495 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:12:29.659309 22505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.640972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41956 (local address 127.10.173.189:39265)
0811 22:12:29.641190 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:29.641199 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:29.641891 (+ 692us) server_negotiation.cc:407] Connection header received
0811 22:12:29.642214 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:29.642221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:29.642316 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:29.642451 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:29.644852 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.645753 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.647136 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:29.647763 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:29.647892 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:29.651882 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:29.651891 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:29.651899 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:29.651945 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:29.655507 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.656350 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.656356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.656359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.656511 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:29.656940 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:29.656946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:29.656950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:29.657318 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:29.657565 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:29.658680 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0811 22:12:29.658930 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:12:30.411374 22598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.410412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.130:40567)
0811 22:12:30.410988 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.411144 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":257,"threads_started":1}
W20260811 22:12:30.411881 22248 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:32997 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:30.416287 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.415365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.131:45987)
0811 22:12:30.415879 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.416021 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":274,"threads_started":1}
W20260811 22:12:30.416771 22483 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:32997 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:30.423013 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.422023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.129:57813)
0811 22:12:30.422633 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.422743 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":333,"threads_started":1}
W20260811 22:12:30.423475 21994 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:32997 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:30.935381 22602 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11e46714a91e4356a74769c0ceaaed9c)
I20260811 22:12:30.935674 22602 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } }
I20260811 22:12:30.936025 22604 raft_consensus.cc:482] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 11e46714a91e4356a74769c0ceaaed9c)
I20260811 22:12:30.936370 22604 raft_consensus.cc:504] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } }
I20260811 22:12:30.936780 22602 leader_election.cc:288] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997), acb8f75a203e4a578b5208089e109afc (127.10.173.189:39265)
I20260811 22:12:30.939021 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.936886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36934)
0811 22:12:30.938322 (+ 1436us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.938602 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1281,"thread_start_us":1123,"threads_started":1}
I20260811 22:12:30.939451 22604 leader_election.cc:288] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997), ff65fc83f0854276a28f78343c6d7d31 (127.10.173.188:37661)
W20260811 22:12:30.939419 21683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:30.941445 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.938651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36938)
0811 22:12:30.940966 (+ 2315us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.941156 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2168,"thread_start_us":1683,"threads_started":1}
W20260811 22:12:30.941851 21373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:30.942317 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.941875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36946)
0811 22:12:30.942062 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.942152 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:12:30.942579 21683 leader_election.cc:334] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997): Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:30.944525 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.943852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36966)
0811 22:12:30.944269 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.944360 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314}
W20260811 22:12:30.944763 21373 leader_election.cc:334] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997): Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:30.948926 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.938802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41884 (local address 127.10.173.188:37661)
0811 22:12:30.940159 (+ 1357us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:30.940167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:30.941042 (+ 875us) server_negotiation.cc:407] Connection header received
0811 22:12:30.941140 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:30.941145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:30.941229 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:30.941325 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:30.942298 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:30.943917 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:30.946638 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:30.947755 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:30.947902 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:30.947955 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:30.948369 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 22:12:30.948465 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1154,"thread_start_us":1019,"threads_started":1}
I20260811 22:12:30.948937 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.938271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37661 (local address 127.0.0.1:41884)
0811 22:12:30.940672 (+ 2401us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.940695 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:30.940891 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:30.941598 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:30.941605 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:30.941636 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:30.942064 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:30.942081 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:30.944090 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:30.944095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:30.946470 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:30.946485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:30.947933 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:30.947944 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:30.948174 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:30.948213 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:12:30.948318 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:12:30.948449 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2254,"mutex_wait_us":24,"thread_start_us":2392,"threads_started":1}
I20260811 22:12:30.949469 22608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.937377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.10.173.189:39265)
0811 22:12:30.938704 (+ 1327us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:30.938712 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:30.939330 (+ 618us) server_negotiation.cc:407] Connection header received
0811 22:12:30.941325 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:30.941331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:30.941403 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:30.941478 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:30.943460 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:30.944844 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:30.947335 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:30.948416 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:30.948598 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:30.948675 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:30.948996 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 22:12:30.949134 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":1165,"thread_start_us":919,"threads_started":1}
I20260811 22:12:30.949524 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.937024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:39265 (local address 127.0.0.1:41960)
0811 22:12:30.939066 (+ 2042us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.939086 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:30.941145 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:30.942116 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:30.942122 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:30.942152 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:30.943207 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:30.943227 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:30.945048 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:30.945055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:30.947186 (+ 2131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:30.947200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:30.948584 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:30.948592 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:30.948802 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:30.948835 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:12:30.948924 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:12:30.949077 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1888,"mutex_wait_us":1,"thread_start_us":1748,"threads_started":1}
I20260811 22:12:30.949777 21721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "acb8f75a203e4a578b5208089e109afc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff65fc83f0854276a28f78343c6d7d31" is_pre_election: true
I20260811 22:12:30.950135 21721 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acb8f75a203e4a578b5208089e109afc in term 1.
I20260811 22:12:30.950187 21416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff65fc83f0854276a28f78343c6d7d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "acb8f75a203e4a578b5208089e109afc" is_pre_election: true
I20260811 22:12:30.950448 21416 raft_consensus.cc:2454] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff65fc83f0854276a28f78343c6d7d31 in term 1.
I20260811 22:12:30.950693 21370 leader_election.cc:302] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: acb8f75a203e4a578b5208089e109afc, ff65fc83f0854276a28f78343c6d7d31; no voters: 11e46714a91e4356a74769c0ceaaed9c
I20260811 22:12:30.950928 21680 leader_election.cc:302] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: acb8f75a203e4a578b5208089e109afc, ff65fc83f0854276a28f78343c6d7d31; no voters: 11e46714a91e4356a74769c0ceaaed9c
I20260811 22:12:30.951153 22604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 22:12:30.951305 22604 raft_consensus.cc:482] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11e46714a91e4356a74769c0ceaaed9c)
I20260811 22:12:30.951350 22602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 22:12:30.951422 22604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:12:30.951488 22602 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 11e46714a91e4356a74769c0ceaaed9c)
I20260811 22:12:30.951584 22602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:12:30.953649 22602 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } }
I20260811 22:12:30.953663 22604 raft_consensus.cc:504] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } }
I20260811 22:12:30.954262 22602 leader_election.cc:288] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 2 election: Requested vote from peers 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997), acb8f75a203e4a578b5208089e109afc (127.10.173.189:39265)
I20260811 22:12:30.954869 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.954404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36974)
0811 22:12:30.954631 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.954734 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 22:12:30.954906 22604 leader_election.cc:288] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [CANDIDATE]: Term 2 election: Requested vote from peers 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997), ff65fc83f0854276a28f78343c6d7d31 (127.10.173.188:37661)
I20260811 22:12:30.954869 21416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff65fc83f0854276a28f78343c6d7d31" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "acb8f75a203e4a578b5208089e109afc"
I20260811 22:12:30.955199 21416 raft_consensus.cc:2379] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ff65fc83f0854276a28f78343c6d7d31 in current term 2: Already voted for candidate acb8f75a203e4a578b5208089e109afc in this term.
I20260811 22:12:30.955313 21721 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "acb8f75a203e4a578b5208089e109afc" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ff65fc83f0854276a28f78343c6d7d31"
I20260811 22:12:30.955608 21721 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate acb8f75a203e4a578b5208089e109afc in current term 2: Already voted for candidate ff65fc83f0854276a28f78343c6d7d31 in this term.
I20260811 22:12:30.954811 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.954373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36978)
0811 22:12:30.954573 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.954642 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 22:12:30.955886 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.955481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:36988)
0811 22:12:30.955658 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.955734 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 22:12:30.956133 21683 leader_election.cc:334] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997): Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:30.956271 21683 leader_election.cc:302] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ff65fc83f0854276a28f78343c6d7d31; no voters: 11e46714a91e4356a74769c0ceaaed9c, acb8f75a203e4a578b5208089e109afc
I20260811 22:12:30.956553 22602 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 22:12:30.956902 22610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.956534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:37004)
0811 22:12:30.956689 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.956781 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 22:12:30.957070 21373 leader_election.cc:334] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997): Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:30.957190 21373 leader_election.cc:302] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acb8f75a203e4a578b5208089e109afc; no voters: 11e46714a91e4356a74769c0ceaaed9c, ff65fc83f0854276a28f78343c6d7d31
I20260811 22:12:30.957445 22604 raft_consensus.cc:2726] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 22:12:31.414014 22616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:31.412827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.130:58465)
0811 22:12:31.413508 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:31.413668 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":379,"threads_started":1}
I20260811 22:12:31.418670 22617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:31.417493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.131:55087)
0811 22:12:31.418276 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:31.418405 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":523,"threads_started":1}
I20260811 22:12:31.425060 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:31.424181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.129:57933)
0811 22:12:31.424755 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:31.424850 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":290,"threads_started":1}
I20260811 22:12:32.415998 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.415047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.130:41627)
0811 22:12:32.415648 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.415766 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":326,"threads_started":1}
I20260811 22:12:32.420702 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.419704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.131:57105)
0811 22:12:32.420252 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.420403 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":311,"threads_started":1}
I20260811 22:12:32.426911 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.426023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.129:43135)
0811 22:12:32.426639 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.426747 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":326,"threads_started":1}
I20260811 22:12:32.623261 22624 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:32.623479 22624 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } }
I20260811 22:12:32.624307 22624 leader_election.cc:288] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997), acb8f75a203e4a578b5208089e109afc (127.10.173.189:39265)
I20260811 22:12:32.625095 21416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff65fc83f0854276a28f78343c6d7d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "acb8f75a203e4a578b5208089e109afc" is_pre_election: true
I20260811 22:12:32.625394 21416 raft_consensus.cc:2454] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ff65fc83f0854276a28f78343c6d7d31 in term 2.
I20260811 22:12:32.625612 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.624396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:41214)
0811 22:12:32.625233 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.625362 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":565,"threads_started":1}
I20260811 22:12:32.625977 21680 leader_election.cc:302] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: acb8f75a203e4a578b5208089e109afc, ff65fc83f0854276a28f78343c6d7d31; no voters:
I20260811 22:12:32.626446 22624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 22:12:32.626642 22624 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:32.626766 22624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:12:32.627285 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.626925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:41224)
0811 22:12:32.627092 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.627172 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 22:12:32.627465 21683 leader_election.cc:334] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997): Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:32.629096 22624 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } }
I20260811 22:12:32.629637 22624 leader_election.cc:288] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 3 election: Requested vote from peers 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997), acb8f75a203e4a578b5208089e109afc (127.10.173.189:39265)
I20260811 22:12:32.630203 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.629761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:41238)
0811 22:12:32.629910 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.630027 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:12:32.630165 21416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ff65fc83f0854276a28f78343c6d7d31" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "acb8f75a203e4a578b5208089e109afc"
I20260811 22:12:32.630373 21416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:12:32.631220 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.630840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:41242)
0811 22:12:32.631001 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.631083 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:12:32.631387 21683 leader_election.cc:334] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997): Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
I20260811 22:12:32.632854 21416 raft_consensus.cc:2454] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ff65fc83f0854276a28f78343c6d7d31 in term 3.
I20260811 22:12:32.633302 21680 leader_election.cc:302] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: acb8f75a203e4a578b5208089e109afc, ff65fc83f0854276a28f78343c6d7d31; no voters: 11e46714a91e4356a74769c0ceaaed9c
I20260811 22:12:32.633631 22624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:12:32.634073 22624 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [term 3 LEADER]: Becoming Leader. State: Replica: ff65fc83f0854276a28f78343c6d7d31, State: Running, Role: LEADER
I20260811 22:12:32.634531 22624 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } }
I20260811 22:12:32.636462 22628 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff65fc83f0854276a28f78343c6d7d31. Latest consensus state: current_term: 3 leader_uuid: "ff65fc83f0854276a28f78343c6d7d31" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:32.636816 22628 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:32.637553 22630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:32.641554 22630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:32.642382 22630 catalog_manager.cc:1259] Loaded cluster ID: 35427e1bb15c4054a01441bf811a0e08
I20260811 22:12:32.642519 22630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:32.643720 22630 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:32.644541 22630 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Loaded TSK: 0
I20260811 22:12:32.645462 22630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:12:32.706737 21703 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:32.714231 21703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:32.718658 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.718142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:41262)
0811 22:12:32.718357 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.718507 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 22:12:32.718735 21416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [term 3 FOLLOWER]: Refusing update from remote peer ff65fc83f0854276a28f78343c6d7d31: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 22:12:32.719542 22628 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 22:12:32.719867 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.719482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:41266)
0811 22:12:32.719633 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.719699 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:12:32.720149 21683 consensus_peers.cc:487] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 -> Peer 11e46714a91e4356a74769c0ceaaed9c (127.10.173.190:32997): Couldn't send request to peer 11e46714a91e4356a74769c0ceaaed9c. Status: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:12:32.722198 22635 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff65fc83f0854276a28f78343c6d7d31. Latest consensus state: current_term: 3 leader_uuid: "ff65fc83f0854276a28f78343c6d7d31" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:32.722502 22635 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:32.724627 22635 sys_catalog.cc:455] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ff65fc83f0854276a28f78343c6d7d31" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:32.725006 22635 sys_catalog.cc:458] T 00000000000000000000000000000000 P acb8f75a203e4a578b5208089e109afc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:32.730728 22648 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ff65fc83f0854276a28f78343c6d7d31" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "11e46714a91e4356a74769c0ceaaed9c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32997 } } peers { permanent_uuid: "acb8f75a203e4a578b5208089e109afc" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 39265 } attrs { promote: false } } peers { permanent_uuid: "ff65fc83f0854276a28f78343c6d7d31" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37661 } attrs { promote: false } } }
I20260811 22:12:32.731113 22648 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:32.748339 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.738744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:36603 (local address 127.0.0.1:37098)
0811 22:12:32.738918 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.738942 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:32.739072 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:32.740121 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:32.740126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:32.740155 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:32.740553 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:32.740571 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.742489 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.742499 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:32.745153 (+ 2654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:32.745174 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.747384 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.747390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:32.747630 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:32.747670 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:12:32.747757 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:12:32.747898 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":96}
I20260811 22:12:32.750392 22649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.739030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37098 (local address 127.10.173.130:36603)
0811 22:12:32.739740 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:32.739749 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:32.739770 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:12:32.739846 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:32.739851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:32.739927 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:32.740717 (+ 790us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:32.740928 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.742232 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.745447 (+ 3215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.747171 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.749515 (+ 2344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:32.749645 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:32.749713 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 22:12:32.749914 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":491,"threads_started":1}
I20260811 22:12:32.751564 22139 tablet_service.cc:1467] Processing CreateTablet for tablet 6522bfe1053141f988c871bee5dfccab (DEFAULT_TABLE table=test-table [id=faf0fb68bb75472cbc2228f2d2d45173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:32.752061 22652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.739428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41574 (local address 127.10.173.129:45861)
0811 22:12:32.740667 (+ 1239us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:32.740677 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:32.741493 (+ 816us) server_negotiation.cc:407] Connection header received
0811 22:12:32.741828 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:32.741837 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:32.741958 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:32.742047 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:32.743291 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.744748 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.748431 (+ 3683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.750169 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.750347 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:32.750490 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:32.751357 (+ 867us) server_negotiation.cc:299] Negotiation successful
0811 22:12:32.751564 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1046,"thread_start_us":927,"threads_started":1}
I20260811 22:12:32.752146 22139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6522bfe1053141f988c871bee5dfccab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:32.753018 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.739104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45861 (local address 127.0.0.1:41574)
0811 22:12:32.741368 (+ 2264us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.741392 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:32.741635 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:32.742344 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:32.742352 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:32.742386 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:32.743007 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:32.743024 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.744989 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.745000 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:32.748182 (+ 3182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:32.748204 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.750824 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.750833 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:32.751091 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:32.751146 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 22:12:32.752273 (+ 1127us) client_negotiation.cc:240] Negotiation successful
0811 22:12:32.752493 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2071,"mutex_wait_us":5,"threads_started":1}
I20260811 22:12:32.756458 21886 tablet_service.cc:1467] Processing CreateTablet for tablet 6522bfe1053141f988c871bee5dfccab (DEFAULT_TABLE table=test-table [id=faf0fb68bb75472cbc2228f2d2d45173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:32.756947 21886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6522bfe1053141f988c871bee5dfccab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:32.759666 22657 tablet_bootstrap.cc:492] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390: Bootstrap starting.
I20260811 22:12:32.761838 22657 tablet_bootstrap.cc:654] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:32.761989 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.740676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:33837 (local address 127.0.0.1:41342)
0811 22:12:32.748657 (+ 7981us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.748679 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:32.748800 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:32.755193 (+ 6393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:32.755201 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:32.755243 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:32.755483 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:32.755498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.756935 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.756942 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:32.759658 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:32.759674 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.761304 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.761308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:32.761429 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:32.761449 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:12:32.761557 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:12:32.761690 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7830,"thread_start_us":641,"threads_started":1}
I20260811 22:12:32.762054 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.741136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41342 (local address 127.10.173.131:33837)
0811 22:12:32.754842 (+ 13706us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:32.754849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:32.754870 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:12:32.754939 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:32.754945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:32.755021 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:32.755149 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:32.755715 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.756766 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.759856 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:32.761008 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:32.761127 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:32.761181 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:32.761588 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 22:12:32.761736 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13583,"thread_start_us":14138,"threads_started":1}
I20260811 22:12:32.763487 22374 tablet_service.cc:1467] Processing CreateTablet for tablet 6522bfe1053141f988c871bee5dfccab (DEFAULT_TABLE table=test-table [id=faf0fb68bb75472cbc2228f2d2d45173]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:32.763784 22661 tablet_bootstrap.cc:492] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: Bootstrap starting.
I20260811 22:12:32.763969 22374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6522bfe1053141f988c871bee5dfccab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:32.764225 22657 tablet_bootstrap.cc:492] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390: No bootstrap required, opened a new log
I20260811 22:12:32.764360 22657 ts_tablet_manager.cc:1397] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 22:12:32.765234 22657 raft_consensus.cc:348] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } }
I20260811 22:12:32.765486 22657 raft_consensus.cc:374] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:32.765583 22657 raft_consensus.cc:729] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c94f10d8f018483fbc787ed2b498e390, State: Initialized, Role: FOLLOWER
I20260811 22:12:32.765725 22661 tablet_bootstrap.cc:654] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:32.765897 22657 consensus_queue.cc:260] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } }
I20260811 22:12:32.766947 22657 ts_tablet_manager.cc:1428] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260811 22:12:32.768141 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.767711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.130:44343)
0811 22:12:32.767892 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.767990 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 22:12:32.770354 22247 heartbeater.cc:503] Master 127.10.173.188:37661 was elected leader, sending a full tablet report...
I20260811 22:12:32.774243 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.773795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.130:57947)
0811 22:12:32.773950 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.774061 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 22:12:32.775672 22661 tablet_bootstrap.cc:492] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: No bootstrap required, opened a new log
I20260811 22:12:32.775934 22661 ts_tablet_manager.cc:1397] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 22:12:32.776839 22661 raft_consensus.cc:348] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } }
I20260811 22:12:32.777031 22661 raft_consensus.cc:374] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:32.777135 22661 raft_consensus.cc:729] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2a43230c04d4b64bd8fd1f05e505914, State: Initialized, Role: FOLLOWER
I20260811 22:12:32.777423 22661 consensus_queue.cc:260] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } }
I20260811 22:12:32.778939 22666 tablet_bootstrap.cc:492] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b: Bootstrap starting.
I20260811 22:12:32.779089 22661 ts_tablet_manager.cc:1428] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:12:32.779685 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.779214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.129:57735)
0811 22:12:32.779374 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.779484 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 22:12:32.781383 22666 tablet_bootstrap.cc:654] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:32.781421 21993 heartbeater.cc:503] Master 127.10.173.188:37661 was elected leader, sending a full tablet report...
I20260811 22:12:32.781484 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.781000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.129:54213)
0811 22:12:32.781215 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.781304 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 22:12:32.783933 22666 tablet_bootstrap.cc:492] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b: No bootstrap required, opened a new log
I20260811 22:12:32.784152 22666 ts_tablet_manager.cc:1397] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 22:12:32.784822 22666 raft_consensus.cc:348] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } }
I20260811 22:12:32.785046 22666 raft_consensus.cc:374] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:32.785182 22666 raft_consensus.cc:729] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7de0600ef3442180518f144d68006b, State: Initialized, Role: FOLLOWER
I20260811 22:12:32.785454 22666 consensus_queue.cc:260] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } }
I20260811 22:12:32.786494 22666 ts_tablet_manager.cc:1428] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 22:12:32.787024 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.786562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.131:34437)
0811 22:12:32.786789 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.786883 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 22:12:32.788054 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.787631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.131:60721)
0811 22:12:32.787809 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.787896 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:12:32.788689 22479 heartbeater.cc:503] Master 127.10.173.188:37661 was elected leader, sending a full tablet report...
W20260811 22:12:32.928573 21998 tablet.cc:2369] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:33.037798 22667 raft_consensus.cc:482] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:33.038139 22667 raft_consensus.cc:504] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } }
I20260811 22:12:33.039872 22667 leader_election.cc:288] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c94f10d8f018483fbc787ed2b498e390 (127.10.173.130:36603), 3d7de0600ef3442180518f144d68006b (127.10.173.131:33837)
I20260811 22:12:33.047657 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.040291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:49983 (local address 127.10.173.131:33837)
0811 22:12:33.040505 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:33.040512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:33.040539 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:12:33.040635 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:33.040644 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:33.040728 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:33.040833 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:33.041487 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.042560 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.045240 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.046504 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.046686 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:33.046770 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:33.047258 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 22:12:33.047391 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:12:33.047672 22649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.040312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:33431 (local address 127.10.173.130:36603)
0811 22:12:33.040507 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:33.040515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:33.041130 (+ 615us) server_negotiation.cc:407] Connection header received
0811 22:12:33.041309 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:33.041313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:33.041368 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:33.041481 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:33.041922 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.042861 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.045269 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.046529 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.046693 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:33.046764 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:33.047295 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 22:12:33.047423 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:12:33.047739 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.040130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:33837 (local address 127.10.173.129:49983)
0811 22:12:33.040305 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.040326 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:33.040465 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:33.040906 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:33.040911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:33.040943 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:33.041337 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:33.041348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.042710 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.042716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:33.045073 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:33.045089 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.046823 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.046832 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:33.047056 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:33.047095 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:12:33.047202 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:12:33.047339 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":63}
I20260811 22:12:33.047897 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.040130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:36603 (local address 127.10.173.129:33431)
0811 22:12:33.041007 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.041028 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:33.041158 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:33.041505 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:33.041508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:33.041528 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:33.041774 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:33.041780 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.042997 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.043002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:33.045076 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:33.045089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.046942 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.046948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:33.047106 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:33.047131 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:12:33.047206 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:12:33.047340 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":193,"thread_start_us":364,"threads_started":1}
I20260811 22:12:33.048787 22403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6522bfe1053141f988c871bee5dfccab" candidate_uuid: "f2a43230c04d4b64bd8fd1f05e505914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d7de0600ef3442180518f144d68006b" is_pre_election: true
I20260811 22:12:33.048786 22169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6522bfe1053141f988c871bee5dfccab" candidate_uuid: "f2a43230c04d4b64bd8fd1f05e505914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c94f10d8f018483fbc787ed2b498e390" is_pre_election: true
I20260811 22:12:33.049059 22403 raft_consensus.cc:2454] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2a43230c04d4b64bd8fd1f05e505914 in term 0.
I20260811 22:12:33.049220 22169 raft_consensus.cc:2454] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2a43230c04d4b64bd8fd1f05e505914 in term 0.
I20260811 22:12:33.049561 21814 leader_election.cc:302] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d7de0600ef3442180518f144d68006b, f2a43230c04d4b64bd8fd1f05e505914; no voters:
I20260811 22:12:33.049993 22667 raft_consensus.cc:2781] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:33.050124 22667 raft_consensus.cc:482] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:33.050242 22667 raft_consensus.cc:3037] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:33.052395 22667 raft_consensus.cc:504] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } }
I20260811 22:12:33.052961 22667 leader_election.cc:288] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [CANDIDATE]: Term 1 election: Requested vote from peers c94f10d8f018483fbc787ed2b498e390 (127.10.173.130:36603), 3d7de0600ef3442180518f144d68006b (127.10.173.131:33837)
I20260811 22:12:33.053354 22169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6522bfe1053141f988c871bee5dfccab" candidate_uuid: "f2a43230c04d4b64bd8fd1f05e505914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c94f10d8f018483fbc787ed2b498e390"
I20260811 22:12:33.053383 22403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6522bfe1053141f988c871bee5dfccab" candidate_uuid: "f2a43230c04d4b64bd8fd1f05e505914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d7de0600ef3442180518f144d68006b"
I20260811 22:12:33.053575 22403 raft_consensus.cc:3037] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:33.053566 22169 raft_consensus.cc:3037] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:33.055639 22403 raft_consensus.cc:2454] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2a43230c04d4b64bd8fd1f05e505914 in term 1.
I20260811 22:12:33.055884 22169 raft_consensus.cc:2454] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2a43230c04d4b64bd8fd1f05e505914 in term 1.
I20260811 22:12:33.056038 21814 leader_election.cc:302] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d7de0600ef3442180518f144d68006b, f2a43230c04d4b64bd8fd1f05e505914; no voters:
I20260811 22:12:33.056355 22667 raft_consensus.cc:2781] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:33.056823 22667 raft_consensus.cc:686] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 1 LEADER]: Becoming Leader. State: Replica: f2a43230c04d4b64bd8fd1f05e505914, State: Running, Role: LEADER
I20260811 22:12:33.057670 22667 consensus_queue.cc:237] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } }
I20260811 22:12:33.057826 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.057401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.129:59599)
0811 22:12:33.057574 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.057690 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 22:12:33.061726 21703 catalog_manager.cc:5495] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2a43230c04d4b64bd8fd1f05e505914 (127.10.173.129). New cstate: current_term: 1 leader_uuid: "f2a43230c04d4b64bd8fd1f05e505914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2a43230c04d4b64bd8fd1f05e505914" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 } health_report { overall_health: UNKNOWN } } }
I20260811 22:12:33.114842 22652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.109689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41582 (local address 127.10.173.129:45861)
0811 22:12:33.109882 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:33.109887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:33.109904 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:12:33.109995 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:33.109999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:33.110085 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:33.110218 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:33.110700 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.111428 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.112767 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.113157 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.113264 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:33.113695 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:33.114279 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:33.114386 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:33.114555 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 22:12:33.114622 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 22:12:33.119268 22403 raft_consensus.cc:1261] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b [term 1 FOLLOWER]: Refusing update from remote peer f2a43230c04d4b64bd8fd1f05e505914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:33.119573 22169 raft_consensus.cc:1261] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390 [term 1 FOLLOWER]: Refusing update from remote peer f2a43230c04d4b64bd8fd1f05e505914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:33.120193 22672 consensus_queue.cc:1035] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d7de0600ef3442180518f144d68006b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:33.120683 22667 consensus_queue.cc:1035] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c94f10d8f018483fbc787ed2b498e390" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:33.123884 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.123250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.131:47191)
0811 22:12:33.123564 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.123685 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260811 22:12:33.124493 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.123351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.130:57979)
0811 22:12:33.124138 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.124287 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":670}
I20260811 22:12:33.128037 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.127483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:41282)
0811 22:12:33.127739 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.127837 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260811 22:12:33.128412 22678 mvcc.cc:205] Tried to move back new op lower bound from 7317448102368436224 to 7317448102128570368. Current Snapshot: MvccSnapshot[applied={T|T < 7317448102368436224}]
I20260811 22:12:33.129474 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.129000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:41292)
0811 22:12:33.129227 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.129324 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 22:12:33.130887 22620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.130373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.130:44775)
0811 22:12:33.130570 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.130688 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 22:12:33.132273 22679 mvcc.cc:205] Tried to move back new op lower bound from 7317448102368436224 to 7317448102128570368. Current Snapshot: MvccSnapshot[applied={T|T < 7317448102368436224}]
I20260811 22:12:33.133257 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.132688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.131:35683)
0811 22:12:33.132987 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.133079 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
I20260811 22:12:33.136720 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.135870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.129:47627)
0811 22:12:33.136449 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.136543 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469}
I20260811 22:12:33.140985 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.140473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.10.173.129:44863)
0811 22:12:33.140673 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.140806 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 22:12:33.148319 21704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:33.148695 21704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:33.154677 21704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff65fc83f0854276a28f78343c6d7d31: Sending DeleteTablet for 3 replicas of tablet 6522bfe1053141f988c871bee5dfccab
I20260811 22:12:33.155723 22139 tablet_service.cc:1514] Processing DeleteTablet for tablet 6522bfe1053141f988c871bee5dfccab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:33 UTC) from {username='oryx', principal='oryx/127.10.173.188@KRBTEST.COM'} at 127.0.0.1:37098
I20260811 22:12:33.156371 21886 tablet_service.cc:1514] Processing DeleteTablet for tablet 6522bfe1053141f988c871bee5dfccab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:33 UTC) from {username='oryx', principal='oryx/127.10.173.188@KRBTEST.COM'} at 127.0.0.1:41574
I20260811 22:12:33.157240 22374 tablet_service.cc:1514] Processing DeleteTablet for tablet 6522bfe1053141f988c871bee5dfccab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:33 UTC) from {username='oryx', principal='oryx/127.10.173.188@KRBTEST.COM'} at 127.0.0.1:41342
I20260811 22:12:33.157281 22694 tablet_replica.cc:330] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: stopping tablet replica
I20260811 22:12:33.157557 22694 raft_consensus.cc:2229] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:33.158380 22694 raft_consensus.cc:2258] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:33.159559 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 21666
I20260811 22:12:33.166527 22697 tablet_replica.cc:330] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b: stopping tablet replica
I20260811 22:12:33.167030 22693 tablet_replica.cc:330] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390: stopping tablet replica
I20260811 22:12:33.167063 22697 raft_consensus.cc:2229] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:33.167270 22693 raft_consensus.cc:2229] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:33.167659 22693 raft_consensus.cc:2258] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:33.167665 22697 raft_consensus.cc:2258] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:33.168511 22694 ts_tablet_manager.cc:1905] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:33.168972 22697 ts_tablet_manager.cc:1905] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:33.169381 22693 ts_tablet_manager.cc:1905] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:33.173476 22697 ts_tablet_manager.cc:1918] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:33.173636 22697 log.cc:1196] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/6522bfe1053141f988c871bee5dfccab
I20260811 22:12:33.174147 22697 ts_tablet_manager.cc:1939] T 6522bfe1053141f988c871bee5dfccab P 3d7de0600ef3442180518f144d68006b: Deleting consensus metadata
I20260811 22:12:33.175477 21683 catalog_manager.cc:4841] TS 3d7de0600ef3442180518f144d68006b (127.10.173.131:33837): tablet 6522bfe1053141f988c871bee5dfccab (table test-table [id=faf0fb68bb75472cbc2228f2d2d45173]) successfully deleted
I20260811 22:12:33.175976 22693 ts_tablet_manager.cc:1918] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:33.176118 22693 log.cc:1196] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/6522bfe1053141f988c871bee5dfccab
I20260811 22:12:33.176699 22693 ts_tablet_manager.cc:1939] T 6522bfe1053141f988c871bee5dfccab P c94f10d8f018483fbc787ed2b498e390: Deleting consensus metadata
I20260811 22:12:33.176980 22694 ts_tablet_manager.cc:1918] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:33.177220 22694 log.cc:1196] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/6522bfe1053141f988c871bee5dfccab
I20260811 22:12:33.177863 22694 ts_tablet_manager.cc:1939] T 6522bfe1053141f988c871bee5dfccab P f2a43230c04d4b64bd8fd1f05e505914: Deleting consensus metadata
I20260811 22:12:33.178004 21681 catalog_manager.cc:4841] TS c94f10d8f018483fbc787ed2b498e390 (127.10.173.130:36603): tablet 6522bfe1053141f988c871bee5dfccab (table test-table [id=faf0fb68bb75472cbc2228f2d2d45173]) successfully deleted
I20260811 22:12:33.180380 21680 catalog_manager.cc:4841] TS f2a43230c04d4b64bd8fd1f05e505914 (127.10.173.129:45861): tablet 6522bfe1053141f988c871bee5dfccab (table test-table [id=faf0fb68bb75472cbc2228f2d2d45173]) successfully deleted
I20260811 22:12:33.189288 22652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.160648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41584 (local address 127.10.173.129:45861)
0811 22:12:33.160901 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:33.160910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:33.174088 (+ 13178us) server_negotiation.cc:407] Connection header received
0811 22:12:33.174519 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:33.174524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:33.174619 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:33.174696 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:33.176774 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.178116 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.179401 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.180054 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.180182 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:33.183032 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:33.183045 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:33.183061 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:33.183102 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:33.186323 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:33.187080 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:33.187087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:33.187089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:33.187250 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:33.187638 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:33.187647 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:33.187649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:33.188009 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:33.188356 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:33.188679 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 22:12:33.188923 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:12:33.190107 21983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:33.294625 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 21666
I20260811 22:12:33.321564 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 22012
I20260811 22:12:33.335430 22649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.322380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.10.173.130:36603)
0811 22:12:33.322624 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:33.322631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:33.322740 (+ 109us) server_negotiation.cc:407] Connection header received
0811 22:12:33.322978 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:33.322982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:33.323060 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:33.323167 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:33.324562 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.325323 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.326687 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.327095 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.327232 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:33.329778 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:33.329794 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:33.329812 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:33.329846 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:33.332573 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:33.333256 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:33.333264 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:33.333266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:33.333396 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:33.333817 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:33.333822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:33.333825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:33.334202 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:33.334577 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:33.334863 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 22:12:33.335106 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 22:12:33.336122 22242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:33.458730 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 22012
I20260811 22:12:33.491894 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 22267
I20260811 22:12:33.506120 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.492655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41352 (local address 127.10.173.131:33837)
0811 22:12:33.492839 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:33.492844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:33.492863 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:12:33.492967 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:33.492971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:33.493064 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:33.493161 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:33.494681 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.495473 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.496681 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:33.497100 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:33.497204 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:33.500002 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:33.500020 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:33.500038 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:33.500117 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:33.503347 (+ 3230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:33.504014 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:33.504022 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:33.504024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:33.504210 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:33.504531 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:33.504534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:33.504536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:33.504883 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:33.505177 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:33.505519 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 22:12:33.505802 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:12:33.506906 22473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:33.572507 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.572041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:41302)
0811 22:12:33.572248 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.572352 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:12:33.573565 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.573212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32997 (local address 127.0.0.1:41304)
0811 22:12:33.573367 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.573454 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:32997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 22:12:33.601712 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 22267
I20260811 22:12:33.628978 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 21166
I20260811 22:12:33.630051 21447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:33.751022 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 21166
I20260811 22:12:33.781852 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 21329
I20260811 22:12:33.783166 21751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:33.908749 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 21329
2026-08-11T22:12:33Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12983 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:34 dist-test-slave-68xk krb5kdc[22710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:34 dist-test-slave-68xk krb5kdc[22710](info): set up 2 sockets
Aug 11 22:12:34 dist-test-slave-68xk krb5kdc[22710](info): commencing operation
krb5kdc: starting...
W20260811 22:12:35.282483 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:12:35 dist-test-slave-68xk krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.190.keytab.
Entry for principal oryx/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.190.keytab.
I20260811 22:12:35.558632 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:46259
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.190.keytab
--principal=oryx/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:46259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:35.902843 22728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:35.903156 22728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:35.903301 22728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:35.910600 22728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:35.910709 22728 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:12:35.910785 22728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:35.910881 22728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:35.910977 22728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:35.911065 22728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:35.921253 22728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:46259
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.190.keytab
--principal=oryx/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:46259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:35.922632 22728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:35.924007 22728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:35.936236 22735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:35.936285 22737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:35.936509 22739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:35.937739 22728 server_base.cc:1034] running on GCE node
I20260811 22:12:35.938714 22728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:35.940253 22728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:35.941514 22728 hybrid_clock.cc:648] HybridClock initialized: now 1786486355941431 us; error 68 us; skew 500 ppm
I20260811 22:12:35.941864 22728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:35 dist-test-slave-68xk krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486355, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:35.950727 22728 init.cc:377] Logged in from keytab as oryx/127.10.173.190@KRBTEST.COM (short username oryx)
I20260811 22:12:35.952441 22728 webserver.cc:460] Webserver started at http://127.10.173.190:38443/ using document root <none> and password file <none>
I20260811 22:12:35.953222 22728 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:35.953365 22728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:35.953715 22728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:35.956991 22728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9"
format_stamp: "Formatted at 2026-08-11 22:12:35 on dist-test-slave-68xk"
server_key: "f89943cb67663c8c933504e42d7cfca7"
server_key_iv: "09ea88a250dd644c357683413045d825"
server_key_version: "encryptionkey@0"
I20260811 22:12:35.957830 22728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9"
format_stamp: "Formatted at 2026-08-11 22:12:35 on dist-test-slave-68xk"
server_key: "f89943cb67663c8c933504e42d7cfca7"
server_key_iv: "09ea88a250dd644c357683413045d825"
server_key_version: "encryptionkey@0"
I20260811 22:12:35.963733 22728 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260811 22:12:35.967381 22764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:35.968818 22728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:35.969009 22728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9"
format_stamp: "Formatted at 2026-08-11 22:12:35 on dist-test-slave-68xk"
server_key: "f89943cb67663c8c933504e42d7cfca7"
server_key_iv: "09ea88a250dd644c357683413045d825"
server_key_version: "encryptionkey@0"
I20260811 22:12:35.969233 22728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:35.988618 22728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:35.993259 22728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:35.993575 22728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:36.018332 22728 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:46259
I20260811 22:12:36.018340 22858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:46259 every 8 connection(s)
I20260811 22:12:36.019881 22728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:36.023778 22862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:36.026841 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 22728
I20260811 22:12:36.027093 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:36.027964 10934 external_mini_cluster.cc:1428] Setting key d2b369e14d4c16a6b91f2ece0756d68d
I20260811 22:12:36.037184 22862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9: Bootstrap starting.
I20260811 22:12:36.041175 22862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:36.042555 22862 log.cc:824] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:12:36 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:36.045536 22862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9: No bootstrap required, opened a new log
I20260811 22:12:36.052455 22866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:36.030212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41832 (local address 127.10.173.190:46259)
0811 22:12:36.031119 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:36.031131 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:36.031176 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:12:36.033036 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:36.033064 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:36.033497 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:36.033870 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:36.035685 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:36.037107 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:36.039047 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:36.039772 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:36.039911 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:36.044994 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:36.045002 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:36.045016 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:36.045063 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:36.048364 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:36.049061 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:36.049068 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:36.049074 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:36.049245 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:36.049651 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:36.049655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:36.049657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:36.050189 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:36.050560 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:36.051023 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 22:12:36.051449 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":463,"threads_started":1}
I20260811 22:12:36.053695 22862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46259 } }
I20260811 22:12:36.054054 22862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:36.054167 22862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ffa02a054cf41fdbe1ee61d69ec3ff9, State: Initialized, Role: FOLLOWER
I20260811 22:12:36.054826 22862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46259 } }
I20260811 22:12:36.055161 22862 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:36.055339 22862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:36.055537 22862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:36.057830 22862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46259 } }
I20260811 22:12:36.058334 22862 leader_election.cc:302] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ffa02a054cf41fdbe1ee61d69ec3ff9; no voters:
I20260811 22:12:36.059001 22862 leader_election.cc:288] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:36.059139 22871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:36.059868 22871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [term 1 LEADER]: Becoming Leader. State: Replica: 4ffa02a054cf41fdbe1ee61d69ec3ff9, State: Running, Role: LEADER
I20260811 22:12:36.060456 22871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46259 } }
I20260811 22:12:36.061231 22862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:36.063340 22873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ffa02a054cf41fdbe1ee61d69ec3ff9. Latest consensus state: current_term: 1 leader_uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46259 } } }
I20260811 22:12:36.063747 22873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:36.063584 22872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ffa02a054cf41fdbe1ee61d69ec3ff9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46259 } } }
I20260811 22:12:36.063841 22872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:36.069988 22878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:36.076476 22878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:36.087620 22878 catalog_manager.cc:1347] Generated new cluster ID: 317701735d084e76b1f6a0f1171a68e6
I20260811 22:12:36.087760 22878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:36.113893 22878 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:36.115940 22878 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:36.131564 22878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9: Generated new TSK 0
I20260811 22:12:36.132507 22878 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab.
Entry for principal oryx/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab.
I20260811 22:12:36.211411 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46259
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab
--principal=oryx/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:36.563678 22902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:36.564009 22902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:36.564175 22902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:36.571753 22902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:36.571911 22902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:36.572049 22902 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:12:36.572187 22902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:12:36.582547 22902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab
--principal=oryx/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.173.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:36.583973 22902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:36.585393 22902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:36.599902 22916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:36.599898 22915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:36.599819 22918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:36.601096 22902 server_base.cc:1034] running on GCE node
I20260811 22:12:36.601780 22902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:36.602936 22902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:36.604238 22902 hybrid_clock.cc:648] HybridClock initialized: now 1786486356604180 us; error 48 us; skew 500 ppm
I20260811 22:12:36.604616 22902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:36 dist-test-slave-68xk krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486356, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:36.608459 22902 init.cc:377] Logged in from keytab as oryx/127.10.173.129@KRBTEST.COM (short username oryx)
I20260811 22:12:36.610103 22902 webserver.cc:460] Webserver started at http://127.10.173.129:39173/ using document root <none> and password file <none>
I20260811 22:12:36.610946 22902 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:36.611086 22902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:36.611440 22902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:36.614583 22902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "506a97c9e03e492d9380756144b70dc9"
format_stamp: "Formatted at 2026-08-11 22:12:36 on dist-test-slave-68xk"
server_key: "87a65a184df96efa652e6cfb6ece09e7"
server_key_iv: "e2943f9f3c4f481d9b6b21f87d15ec8e"
server_key_version: "encryptionkey@0"
I20260811 22:12:36.615449 22902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "506a97c9e03e492d9380756144b70dc9"
format_stamp: "Formatted at 2026-08-11 22:12:36 on dist-test-slave-68xk"
server_key: "87a65a184df96efa652e6cfb6ece09e7"
server_key_iv: "e2943f9f3c4f481d9b6b21f87d15ec8e"
server_key_version: "encryptionkey@0"
I20260811 22:12:36.621402 22902 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:12:36.625810 22931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:36.627365 22902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:36.627545 22902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "506a97c9e03e492d9380756144b70dc9"
format_stamp: "Formatted at 2026-08-11 22:12:36 on dist-test-slave-68xk"
server_key: "87a65a184df96efa652e6cfb6ece09e7"
server_key_iv: "e2943f9f3c4f481d9b6b21f87d15ec8e"
server_key_version: "encryptionkey@0"
I20260811 22:12:36.627779 22902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:36.653790 22902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:36.658595 22902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:36.658900 22902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:36.661702 22902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:36.661792 22902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:36.661952 22902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:36.662024 22902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:12:36 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486356, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:36.693965 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:36.662792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:41943 (local address 127.10.173.190:46259)
0811 22:12:36.663342 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:36.663351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:36.664704 (+ 1353us) server_negotiation.cc:407] Connection header received
0811 22:12:36.667614 (+ 2910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:36.667622 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:36.667729 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:36.667865 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:36.670753 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:36.671966 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:36.674646 (+ 2680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:36.675464 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:36.675636 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:36.682448 (+ 6812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:36.682461 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:36.682469 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:36.682532 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:36.686941 (+ 4409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:36.688274 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:36.688282 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:36.688288 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:36.688481 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:36.689233 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:36.689239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:36.689243 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:36.689696 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:36.689866 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:36.693065 (+ 3199us) server_negotiation.cc:299] Negotiation successful
0811 22:12:36.693399 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":260,"threads_started":1}
I20260811 22:12:36.695434 22956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:36.663538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46259 (local address 127.10.173.129:41943)
0811 22:12:36.664510 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:36.664560 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:36.667276 (+ 2716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:36.668220 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:36.668250 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:36.669303 (+ 1053us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:36.670487 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:36.670528 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:36.672241 (+ 1713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:36.672247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:36.674407 (+ 2160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:36.674453 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:36.676116 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:36.676131 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:36.676532 (+ 401us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:36.677538 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:36.677578 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:36.682001 (+ 4423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:36.687096 (+ 5095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:36.687108 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:36.687132 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:36.688070 (+ 938us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:36.688647 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:36.688655 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:36.688662 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:36.689043 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:36.689956 (+ 913us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:36.689970 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:36.690679 (+ 709us) client_negotiation.cc:769] Sending connection context
0811 22:12:36.691125 (+ 446us) client_negotiation.cc:240] Negotiation successful
0811 22:12:36.691931 (+ 806us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":407,"threads_started":1}
I20260811 22:12:36.707839 22902 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:38101
I20260811 22:12:36.707860 23124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:38101 every 8 connection(s)
I20260811 22:12:36.709151 22902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:36.711454 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 22902
I20260811 22:12:36.711714 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:12:36.711907 23125 heartbeater.cc:347] Connected to a master server at 127.10.173.190:46259
I20260811 22:12:36.712164 23125 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:36.712809 10934 external_mini_cluster.cc:1428] Setting key ad8c703267d344d04f0446d144e423cd
I20260811 22:12:36.713039 23125 heartbeater.cc:511] Master 127.10.173.190:46259 requested a full tablet report, sending...
I20260811 22:12:36.715907 22791 ts_manager.cc:194] Registered new tserver with Master: 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101)
I20260811 22:12:36.718132 22791 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:41943
I20260811 22:12:36.732009 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:36.716649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:40475 (local address 127.10.173.190:46259)
0811 22:12:36.716834 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:36.716842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:36.717517 (+ 675us) server_negotiation.cc:407] Connection header received
0811 22:12:36.717704 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:36.717712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:36.717805 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:36.717916 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:36.719089 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:36.719989 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:36.722086 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:36.722687 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:36.722806 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:36.725245 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:36.725250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:36.725254 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:36.725298 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:36.728823 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:36.729712 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:36.729717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:36.729719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:36.729853 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:36.730255 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:36.730258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:36.730260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:36.730635 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:36.730764 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:36.731256 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 22:12:36.731566 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:12:36.732334 23135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:36.716734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46259 (local address 127.10.173.129:40475)
0811 22:12:36.717388 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:36.717410 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:36.717563 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:36.717973 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:36.717978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:36.718606 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:36.718939 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:36.718955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:36.720167 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:36.720176 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:36.721902 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:36.721922 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:36.722887 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:36.722895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:36.723116 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:36.723842 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:36.723871 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:36.724990 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:36.728966 (+ 3976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:36.728998 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:36.729004 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:36.729588 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:36.729968 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:36.729975 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:36.729982 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:36.730147 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:36.730754 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:36.730762 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:36.730982 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 22:12:36.731171 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 22:12:36.731535 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":3,"thread_start_us":345,"threads_started":1}
WARNING: no policy specified for oryx/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab.
Entry for principal oryx/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab.
I20260811 22:12:36.779747 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46259
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab
--principal=oryx/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:37.141076 23140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:37.141353 23140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:37.141510 23140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:37.149044 23140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:37.149206 23140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:37.149374 23140 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:12:37.149505 23140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:12:37.159766 23140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab
--principal=oryx/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.173.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:37.161101 23140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:37.162575 23140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:37.177634 23149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:37.177760 23154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:37.177757 23148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:37.178952 23140 server_base.cc:1034] running on GCE node
I20260811 22:12:37.179603 23140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:37.180563 23140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:37.181879 23140 hybrid_clock.cc:648] HybridClock initialized: now 1786486357181795 us; error 48 us; skew 500 ppm
I20260811 22:12:37.182219 23140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:37 dist-test-slave-68xk krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486357, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:37.185712 23140 init.cc:377] Logged in from keytab as oryx/127.10.173.130@KRBTEST.COM (short username oryx)
I20260811 22:12:37.187703 23140 webserver.cc:460] Webserver started at http://127.10.173.130:44171/ using document root <none> and password file <none>
I20260811 22:12:37.188525 23140 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:37.188695 23140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:37.189076 23140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:37.192431 23140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "f923495ebd9945979ae679a6409be9b9"
format_stamp: "Formatted at 2026-08-11 22:12:37 on dist-test-slave-68xk"
server_key: "c9319608e35dfa40aba1f4ca37a3e81f"
server_key_iv: "fcace9bea7baf4253eb39f676f02f106"
server_key_version: "encryptionkey@0"
I20260811 22:12:37.193337 23140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "f923495ebd9945979ae679a6409be9b9"
format_stamp: "Formatted at 2026-08-11 22:12:37 on dist-test-slave-68xk"
server_key: "c9319608e35dfa40aba1f4ca37a3e81f"
server_key_iv: "fcace9bea7baf4253eb39f676f02f106"
server_key_version: "encryptionkey@0"
I20260811 22:12:37.199401 23140 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 22:12:37.203181 23171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:37.204523 23140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:37.204737 23140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "f923495ebd9945979ae679a6409be9b9"
format_stamp: "Formatted at 2026-08-11 22:12:37 on dist-test-slave-68xk"
server_key: "c9319608e35dfa40aba1f4ca37a3e81f"
server_key_iv: "fcace9bea7baf4253eb39f676f02f106"
server_key_version: "encryptionkey@0"
I20260811 22:12:37.204969 23140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:37.232744 23140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:37.237116 23140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:37.237409 23140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:37.240361 23140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:37.240473 23140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:37.240657 23140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:37.240743 23140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:12:37 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486357, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:37.264334 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.240720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:53991 (local address 127.10.173.190:46259)
0811 22:12:37.242460 (+ 1740us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:37.242467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:37.242705 (+ 238us) server_negotiation.cc:407] Connection header received
0811 22:12:37.244780 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:37.244786 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:37.244883 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:37.245003 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:37.247309 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.248430 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.251501 (+ 3071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.251926 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.252071 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.256573 (+ 4502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:37.256578 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:37.256581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:37.256615 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:37.259809 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:37.260934 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:37.260942 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:37.260947 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:37.261139 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:37.261833 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:37.261838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:37.261843 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:37.262184 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:37.262363 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:37.263413 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0811 22:12:37.263758 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":243,"threads_started":1}
I20260811 22:12:37.265017 23188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.241487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46259 (local address 127.10.173.130:53991)
0811 22:12:37.242527 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:37.242582 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:37.244476 (+ 1894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:37.245261 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:37.245272 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:37.246023 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:37.247048 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.247076 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.248668 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.248673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.251267 (+ 2594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.251290 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.252197 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.252208 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.252547 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.253408 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:37.253447 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:37.256338 (+ 2891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:37.259973 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:37.259982 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:37.260000 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:37.260722 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:37.261350 (+ 628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:37.261356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:37.261362 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:37.261702 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:37.262348 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:37.262358 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:37.262912 (+ 554us) client_negotiation.cc:769] Sending connection context
0811 22:12:37.263256 (+ 344us) client_negotiation.cc:240] Negotiation successful
0811 22:12:37.263718 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":426,"threads_started":1}
I20260811 22:12:37.290359 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.273689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:57005 (local address 127.10.173.190:46259)
0811 22:12:37.273957 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:37.273967 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:37.274693 (+ 726us) server_negotiation.cc:407] Connection header received
0811 22:12:37.274976 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:37.274985 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:37.275083 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:37.275209 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:37.276184 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.277502 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.279785 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.280465 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.280616 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.283628 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:37.283637 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:37.283644 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:37.283699 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:37.287061 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:37.288140 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:37.288145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:37.288147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:37.288296 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:37.288747 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:37.288752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:37.288754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:37.289108 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:37.289276 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:37.289755 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 22:12:37.289993 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:12:37.290566 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.273850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46259 (local address 127.10.173.130:57005)
0811 22:12:37.274584 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:37.274603 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:37.274783 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:37.275266 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:37.275271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:37.275754 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:37.276027 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.276036 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.277678 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.277687 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.279495 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.279513 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.280701 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.280706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.280835 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.281608 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:37.281642 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:37.283366 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:37.287229 (+ 3863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:37.287238 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:37.287244 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:37.287966 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:37.288447 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:37.288453 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:37.288457 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:37.288610 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:37.289237 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:37.289243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:37.289451 (+ 208us) client_negotiation.cc:769] Sending connection context
0811 22:12:37.289624 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 22:12:37.289989 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":380,"threads_started":1}
I20260811 22:12:37.296325 23140 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:42959
I20260811 22:12:37.296419 23391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:42959 every 8 connection(s)
I20260811 22:12:37.297643 23140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:37.299993 23392 heartbeater.cc:347] Connected to a master server at 127.10.173.190:46259
I20260811 22:12:37.300257 23392 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:37.301275 23392 heartbeater.cc:511] Master 127.10.173.190:46259 requested a full tablet report, sending...
I20260811 22:12:37.301693 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 23140
I20260811 22:12:37.301916 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:12:37.302551 10934 external_mini_cluster.cc:1428] Setting key e31bbc22c977d06a818bdee01d89c235
I20260811 22:12:37.303171 22791 ts_manager.cc:194] Registered new tserver with Master: f923495ebd9945979ae679a6409be9b9 (127.10.173.130:42959)
I20260811 22:12:37.304255 22791 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:53991
WARNING: no policy specified for oryx/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab.
Entry for principal oryx/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab.
I20260811 22:12:37.366731 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46259
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab
--principal=oryx/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:40739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:37.715569 23397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:37.715876 23397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:37.716049 23397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:12:37.721621 23125 heartbeater.cc:503] Master 127.10.173.190:46259 was elected leader, sending a full tablet report...
W20260811 22:12:37.723724 23397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:37.723870 23397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:37.724007 23397 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:12:37.724148 23397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:12:37.734215 23397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:40739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab
--principal=oryx/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.173.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:37.735565 23397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:37.736928 23397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:37.750936 23414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:37.751025 23407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:37.750963 23408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:37.751709 23397 server_base.cc:1034] running on GCE node
I20260811 22:12:37.752466 23397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:37.753563 23397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:37.754878 23397 hybrid_clock.cc:648] HybridClock initialized: now 1786486357754777 us; error 57 us; skew 500 ppm
I20260811 22:12:37.755188 23397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:37 dist-test-slave-68xk krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486357, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:37.758952 23397 init.cc:377] Logged in from keytab as oryx/127.10.173.131@KRBTEST.COM (short username oryx)
I20260811 22:12:37.760785 23397 webserver.cc:460] Webserver started at http://127.10.173.131:43191/ using document root <none> and password file <none>
I20260811 22:12:37.761713 23397 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:37.761875 23397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:37.762251 23397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:37.765806 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "f76309604a1e459eaa85ff839797bd07"
format_stamp: "Formatted at 2026-08-11 22:12:37 on dist-test-slave-68xk"
server_key: "7d52f663692b252045201c976433ec99"
server_key_iv: "aa1b16818648c4eebbc4a94b1e1f3df9"
server_key_version: "encryptionkey@0"
I20260811 22:12:37.766856 23397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "f76309604a1e459eaa85ff839797bd07"
format_stamp: "Formatted at 2026-08-11 22:12:37 on dist-test-slave-68xk"
server_key: "7d52f663692b252045201c976433ec99"
server_key_iv: "aa1b16818648c4eebbc4a94b1e1f3df9"
server_key_version: "encryptionkey@0"
I20260811 22:12:37.773288 23397 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:12:37.777112 23425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:37.778745 23397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:37.778937 23397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "f76309604a1e459eaa85ff839797bd07"
format_stamp: "Formatted at 2026-08-11 22:12:37 on dist-test-slave-68xk"
server_key: "7d52f663692b252045201c976433ec99"
server_key_iv: "aa1b16818648c4eebbc4a94b1e1f3df9"
server_key_version: "encryptionkey@0"
I20260811 22:12:37.779171 23397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:37.790091 23397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:37.794852 23397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:37.795183 23397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:37.797983 23397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:37.798089 23397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:37.798224 23397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:37.798302 23397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:12:37 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486357, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:37.823640 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.798652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:55777 (local address 127.10.173.190:46259)
0811 22:12:37.801232 (+ 2580us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:37.801240 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:37.801266 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:12:37.803019 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:37.803025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:37.803112 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:37.803263 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:37.806457 (+ 3194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.807474 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.810697 (+ 3223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.811293 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.811439 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.816330 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:37.816335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:37.816339 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:37.816372 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:37.819351 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:37.820655 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:37.820663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:37.820669 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:37.820828 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:37.821444 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:37.821452 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:37.821456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:37.821945 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:37.822159 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:37.822972 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 22:12:37.823228 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2440,"thread_start_us":2309,"threads_started":1}
I20260811 22:12:37.824704 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.799371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46259 (local address 127.10.173.131:55777)
0811 22:12:37.800469 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:37.800531 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:37.802720 (+ 2189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:37.803819 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:37.803833 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:37.804983 (+ 1150us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:37.806183 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.806215 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.807687 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.807696 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.810479 (+ 2783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.810503 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.811592 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.811602 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.811984 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.812938 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:37.812977 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:37.816094 (+ 3117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:37.819541 (+ 3447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:37.819553 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:37.819571 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:37.820478 (+ 907us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:37.820953 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:37.820958 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:37.820962 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:37.821230 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:37.822100 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:37.822107 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:37.822542 (+ 435us) client_negotiation.cc:769] Sending connection context
0811 22:12:37.822871 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 22:12:37.823456 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":416,"threads_started":1}
I20260811 22:12:37.847661 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.831468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46259 (local address 127.10.173.131:55033)
0811 22:12:37.832193 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:37.832208 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:37.832343 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:37.832804 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:37.832807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:37.833243 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:37.833529 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.833537 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.835331 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.835336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.836341 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.836351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.837606 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.837610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.837749 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.838477 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:37.838495 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:37.839721 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:37.844578 (+ 4857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:37.844585 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:37.844589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:37.845119 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:37.845635 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:37.845640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:37.845643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:37.845842 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:37.846542 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:37.846545 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:37.846714 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 22:12:37.846849 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 22:12:37.847096 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":397,"threads_started":1}
I20260811 22:12:37.847710 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.831322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:55033 (local address 127.10.173.190:46259)
0811 22:12:37.831577 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:37.831584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:37.832291 (+ 707us) server_negotiation.cc:407] Connection header received
0811 22:12:37.832484 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:37.832493 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:37.832598 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:37.832762 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:37.833678 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.835151 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.836484 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.837408 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.837574 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.840009 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:37.840017 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:37.840021 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:37.840063 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:37.844413 (+ 4350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:37.845321 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:37.845327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:37.845330 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:37.845499 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:37.845984 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:37.845989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:37.845992 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:37.846386 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:37.846587 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:37.846991 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 22:12:37.847264 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:12:37.855568 23397 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:39483
I20260811 22:12:37.855610 23619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:39483 every 8 connection(s)
I20260811 22:12:37.856815 23397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:37.858958 23625 heartbeater.cc:347] Connected to a master server at 127.10.173.190:46259
I20260811 22:12:37.859189 23625 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:37.859881 23625 heartbeater.cc:511] Master 127.10.173.190:46259 requested a full tablet report, sending...
I20260811 22:12:37.861268 22791 ts_manager.cc:194] Registered new tserver with Master: f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483)
I20260811 22:12:37.862229 22791 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:55777
I20260811 22:12:37.864964 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 23397
I20260811 22:12:37.865180 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:37.865605 10934 external_mini_cluster.cc:1428] Setting key 5778dc4943010f0a6f0a36bd4e19c6b3
I20260811 22:12:37.868696 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:12:37 dist-test-slave-68xk krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:12:37 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:37 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:12:37.930423 23438 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:37.919031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.10.173.190:46259)
0811 22:12:37.919280 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:37.919286 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:37.919418 (+ 132us) server_negotiation.cc:407] Connection header received
0811 22:12:37.919693 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:37.919697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:37.919782 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:37.919903 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:37.921342 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.922078 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.923526 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.923980 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.924166 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.930142 (+ 5976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41840: BlockingRecv error: recv got EOF from 127.0.0.1:41840 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 11 22:12:37 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486357, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:37.948449 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.934665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41842 (local address 127.10.173.190:46259)
0811 22:12:37.934901 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:37.934910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:37.935212 (+ 302us) server_negotiation.cc:407] Connection header received
0811 22:12:37.935430 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:37.935439 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:37.935504 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:37.935613 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:37.936847 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.937647 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.939008 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.939443 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.939584 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.943388 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:37.943395 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:37.943398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:37.943432 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:37.946030 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:37.946767 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:37.946772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:37.946774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:37.946902 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:37.947281 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:37.947284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:37.947286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:37.947545 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:37.947737 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:37.948011 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 22:12:37.948211 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:12:37.952929 22791 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:37.960399 22791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:37.984817 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.973342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:39483 (local address 127.0.0.1:52910)
0811 22:12:37.975061 (+ 1719us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:37.975112 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:37.975282 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:37.976835 (+ 1553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:37.976843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:37.976875 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:37.977379 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.977396 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.979611 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.979616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.981765 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.981777 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.983596 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.983604 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.983861 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.983922 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 22:12:37.984055 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 22:12:37.984324 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1464,"mutex_wait_us":27,"thread_start_us":3906,"threads_started":1}
I20260811 22:12:37.985759 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.973543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52910 (local address 127.10.173.131:39483)
0811 22:12:37.976122 (+ 2579us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:37.976131 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:37.976170 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:12:37.976266 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:37.976274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:37.976562 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:37.976830 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:37.977727 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.979468 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.981934 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.983370 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.983581 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.983770 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:37.984118 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 22:12:37.984302 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2375,"thread_start_us":2125,"threads_started":1}
I20260811 22:12:37.985975 23650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.973797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:38101 (local address 127.0.0.1:38244)
0811 22:12:37.977603 (+ 3806us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:37.977622 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:37.977749 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:37.978558 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:37.978562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:37.978587 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:37.979131 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.979150 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.980923 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.980931 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.982988 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.983000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.985201 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.985210 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.985420 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.985456 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:12:37.985539 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:12:37.985661 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3662,"thread_start_us":871,"threads_started":1}
I20260811 22:12:37.986110 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.974440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38244 (local address 127.10.173.129:38101)
0811 22:12:37.976411 (+ 1971us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:37.976423 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:37.977712 (+ 1289us) server_negotiation.cc:407] Connection header received
0811 22:12:37.977893 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:37.977901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:37.978139 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:37.978331 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:37.979321 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.980752 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.983156 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.984645 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.984829 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.985007 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:37.985628 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 22:12:37.985811 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1760,"thread_start_us":1597,"threads_started":1}
I20260811 22:12:37.988209 23657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.978480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:42959 (local address 127.0.0.1:46010)
0811 22:12:37.979154 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:37.979168 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:37.979312 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:37.979946 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:37.979951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:37.979975 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:37.980604 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.980618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.982457 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.982462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.984763 (+ 2301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:37.984775 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.987370 (+ 2595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.987376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:37.987547 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.987578 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:12:37.987676 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 22:12:37.987817 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":4,"thread_start_us":389,"threads_started":1}
I20260811 22:12:37.988217 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.977911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46010 (local address 127.10.173.130:42959)
0811 22:12:37.978470 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:37.978479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:37.979275 (+ 796us) server_negotiation.cc:407] Connection header received
0811 22:12:37.979491 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:37.979505 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:37.979729 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:37.979938 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:37.980826 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.982280 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.984963 (+ 2683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:37.987151 (+ 2188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:37.987350 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:37.987525 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:37.987749 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 22:12:37.987929 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":244,"threads_started":1}
I20260811 22:12:37.989923 23024 tablet_service.cc:1467] Processing CreateTablet for tablet e6618a90e1f84ca5b46223502f2b84b3 (DEFAULT_TABLE table=test-table [id=d06ffd62d2d44f2fa74e7cc9d489d809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:37.989866 23508 tablet_service.cc:1467] Processing CreateTablet for tablet e6618a90e1f84ca5b46223502f2b84b3 (DEFAULT_TABLE table=test-table [id=d06ffd62d2d44f2fa74e7cc9d489d809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:37.991384 23024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6618a90e1f84ca5b46223502f2b84b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:37.991581 23508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6618a90e1f84ca5b46223502f2b84b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:37.991638 23267 tablet_service.cc:1467] Processing CreateTablet for tablet e6618a90e1f84ca5b46223502f2b84b3 (DEFAULT_TABLE table=test-table [id=d06ffd62d2d44f2fa74e7cc9d489d809]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:37.992903 23267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6618a90e1f84ca5b46223502f2b84b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:38.004859 23658 tablet_bootstrap.cc:492] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: Bootstrap starting.
I20260811 22:12:38.006908 23659 tablet_bootstrap.cc:492] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: Bootstrap starting.
I20260811 22:12:38.008935 23658 tablet_bootstrap.cc:654] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:38.010303 23658 log.cc:824] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:38.010593 23659 tablet_bootstrap.cc:654] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:38.011901 23659 log.cc:824] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:38.014397 23658 tablet_bootstrap.cc:492] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: No bootstrap required, opened a new log
I20260811 22:12:38.014669 23660 tablet_bootstrap.cc:492] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: Bootstrap starting.
I20260811 22:12:38.014775 23658 ts_tablet_manager.cc:1397] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260811 22:12:38.017277 23660 tablet_bootstrap.cc:654] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:38.018613 23660 log.cc:824] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:38.020157 23659 tablet_bootstrap.cc:492] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: No bootstrap required, opened a new log
I20260811 22:12:38.020450 23659 ts_tablet_manager.cc:1397] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260811 22:12:38.021234 23660 tablet_bootstrap.cc:492] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: No bootstrap required, opened a new log
I20260811 22:12:38.021492 23660 ts_tablet_manager.cc:1397] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 22:12:38.026253 23658 raft_consensus.cc:348] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } }
I20260811 22:12:38.026738 23658 raft_consensus.cc:374] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:38.026860 23658 raft_consensus.cc:729] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a97c9e03e492d9380756144b70dc9, State: Initialized, Role: FOLLOWER
I20260811 22:12:38.027623 23658 consensus_queue.cc:260] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } }
I20260811 22:12:38.031956 23660 raft_consensus.cc:348] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } }
I20260811 22:12:38.032452 23660 raft_consensus.cc:374] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:38.032598 23660 raft_consensus.cc:729] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f923495ebd9945979ae679a6409be9b9, State: Initialized, Role: FOLLOWER
I20260811 22:12:38.032444 23659 raft_consensus.cc:348] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } }
I20260811 22:12:38.033012 23659 raft_consensus.cc:374] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:38.033196 23659 raft_consensus.cc:729] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76309604a1e459eaa85ff839797bd07, State: Initialized, Role: FOLLOWER
I20260811 22:12:38.033272 23660 consensus_queue.cc:260] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } }
I20260811 22:12:38.034053 23659 consensus_queue.cc:260] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } }
I20260811 22:12:38.036613 23658 ts_tablet_manager.cc:1428] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: Time spent starting tablet: real 0.022s user 0.015s sys 0.004s
I20260811 22:12:38.038427 23625 heartbeater.cc:503] Master 127.10.173.190:46259 was elected leader, sending a full tablet report...
I20260811 22:12:38.039407 23659 ts_tablet_manager.cc:1428] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: Time spent starting tablet: real 0.019s user 0.013s sys 0.004s
I20260811 22:12:38.040261 23392 heartbeater.cc:503] Master 127.10.173.190:46259 was elected leader, sending a full tablet report...
I20260811 22:12:38.041146 23660 ts_tablet_manager.cc:1428] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: Time spent starting tablet: real 0.020s user 0.016s sys 0.000s
W20260811 22:12:38.049849 23393 tablet.cc:2369] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:12:38.108471 23626 tablet.cc:2369] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:12:38.212049 23126 tablet.cc:2369] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:38.322247 23668 raft_consensus.cc:482] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:38.322669 23668 raft_consensus.cc:504] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } }
I20260811 22:12:38.324733 23668 leader_election.cc:288] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483), 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101)
I20260811 22:12:38.335320 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.325375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:44313 (local address 127.10.173.129:38101)
0811 22:12:38.325790 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:38.325800 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:38.326772 (+ 972us) server_negotiation.cc:407] Connection header received
0811 22:12:38.327362 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:38.327366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:38.327426 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:38.327495 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:38.328184 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.329247 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.332522 (+ 3275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.333824 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.333989 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.334066 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:38.334909 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 22:12:38.335050 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214}
I20260811 22:12:38.335320 23651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.325318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:36001 (local address 127.10.173.131:39483)
0811 22:12:38.325650 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:38.325663 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:38.326669 (+ 1006us) server_negotiation.cc:407] Connection header received
0811 22:12:38.326955 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:38.326959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:38.327083 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:38.327168 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:38.327931 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.329056 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.332551 (+ 3495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.333824 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.333991 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.334066 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:38.334909 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 22:12:38.335049 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260811 22:12:38.335539 23670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.325314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:38101 (local address 127.10.173.130:44313)
0811 22:12:38.326509 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:38.326541 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:38.326757 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:38.327559 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:38.327565 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:38.327605 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:38.328029 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:38.328038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.329373 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.329378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:38.332298 (+ 2920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:38.332316 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.334239 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.334254 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:38.334583 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.334718 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 22:12:38.334837 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 22:12:38.335068 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":989,"mutex_wait_us":239,"thread_start_us":527,"threads_started":1}
I20260811 22:12:38.336022 23671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.325314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:39483 (local address 127.10.173.130:36001)
0811 22:12:38.326517 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:38.326534 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:38.326779 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:38.327220 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:38.327224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:38.327266 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:38.327710 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:38.327727 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.329200 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.329204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:38.332298 (+ 3094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:38.332315 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.334268 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.334274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:38.334590 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.334718 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 22:12:38.334837 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 22:12:38.335068 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060,"mutex_wait_us":418,"spinlock_wait_cycles":86272,"thread_start_us":461,"threads_started":1}
I20260811 22:12:38.336751 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6618a90e1f84ca5b46223502f2b84b3" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a97c9e03e492d9380756144b70dc9" is_pre_election: true
I20260811 22:12:38.336931 23545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6618a90e1f84ca5b46223502f2b84b3" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76309604a1e459eaa85ff839797bd07" is_pre_election: true
I20260811 22:12:38.337348 23054 raft_consensus.cc:2454] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 0.
I20260811 22:12:38.337410 23545 raft_consensus.cc:2454] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 0.
I20260811 22:12:38.338058 23172 leader_election.cc:302] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506a97c9e03e492d9380756144b70dc9, f923495ebd9945979ae679a6409be9b9; no voters:
I20260811 22:12:38.338514 23668 raft_consensus.cc:2781] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:38.338693 23668 raft_consensus.cc:482] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:38.338819 23668 raft_consensus.cc:3037] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:38.341161 23668 raft_consensus.cc:504] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } }
I20260811 22:12:38.341820 23668 leader_election.cc:288] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 election: Requested vote from peers f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483), 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101)
I20260811 22:12:38.342283 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6618a90e1f84ca5b46223502f2b84b3" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a97c9e03e492d9380756144b70dc9"
I20260811 22:12:38.342516 23054 raft_consensus.cc:3037] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:38.342473 23545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6618a90e1f84ca5b46223502f2b84b3" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76309604a1e459eaa85ff839797bd07"
I20260811 22:12:38.342703 23545 raft_consensus.cc:3037] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:38.344971 23054 raft_consensus.cc:2454] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 1.
I20260811 22:12:38.345534 23545 raft_consensus.cc:2454] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 1.
I20260811 22:12:38.345743 23172 leader_election.cc:302] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506a97c9e03e492d9380756144b70dc9, f923495ebd9945979ae679a6409be9b9; no voters:
I20260811 22:12:38.346065 23668 raft_consensus.cc:2781] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:38.346607 23668 raft_consensus.cc:686] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 1 LEADER]: Becoming Leader. State: Replica: f923495ebd9945979ae679a6409be9b9, State: Running, Role: LEADER
I20260811 22:12:38.347090 23668 consensus_queue.cc:237] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } }
I20260811 22:12:38.352224 22791 catalog_manager.cc:5495] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f923495ebd9945979ae679a6409be9b9 (127.10.173.130). New cstate: current_term: 1 leader_uuid: "f923495ebd9945979ae679a6409be9b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } health_report { overall_health: UNKNOWN } } }
I20260811 22:12:38.446940 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.441564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.10.173.130:42959)
0811 22:12:38.441756 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:38.441763 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:38.441864 (+ 101us) server_negotiation.cc:407] Connection header received
0811 22:12:38.442054 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:38.442057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:38.442139 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:38.442273 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:38.442665 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.443381 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.444787 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.445195 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.445298 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.445712 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:38.446246 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:38.446395 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:38.446658 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 22:12:38.446728 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:12:38.455332 23054 raft_consensus.cc:1261] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Refusing update from remote peer f923495ebd9945979ae679a6409be9b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:38.455358 23545 raft_consensus.cc:1261] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Refusing update from remote peer f923495ebd9945979ae679a6409be9b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:38.456385 23672 consensus_queue.cc:1035] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:38.456758 23668 consensus_queue.cc:1035] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:38.467633 23680 mvcc.cc:205] Tried to move back new op lower bound from 7317448124221984768 to 7317448123793469440. Current Snapshot: MvccSnapshot[applied={T|T < 7317448124221984768}]
I20260811 22:12:38.473371 23682 mvcc.cc:205] Tried to move back new op lower bound from 7317448124221984768 to 7317448123793469440. Current Snapshot: MvccSnapshot[applied={T|T < 7317448124221984768}]
I20260811 22:12:38.484375 22791 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:38.484627 22791 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:38.488396 22791 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9: Sending DeleteTablet for 3 replicas of tablet e6618a90e1f84ca5b46223502f2b84b3
I20260811 22:12:38.489516 23508 tablet_service.cc:1514] Processing DeleteTablet for tablet e6618a90e1f84ca5b46223502f2b84b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:38 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:52910
I20260811 22:12:38.489601 23267 tablet_service.cc:1514] Processing DeleteTablet for tablet e6618a90e1f84ca5b46223502f2b84b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:38 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:46010
I20260811 22:12:38.490346 23024 tablet_service.cc:1514] Processing DeleteTablet for tablet e6618a90e1f84ca5b46223502f2b84b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:38 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:38244
I20260811 22:12:38.490496 23698 tablet_replica.cc:330] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: stopping tablet replica
I20260811 22:12:38.491024 23698 raft_consensus.cc:2229] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:38.491590 23698 raft_consensus.cc:2258] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:38.491621 23699 tablet_replica.cc:330] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: stopping tablet replica
I20260811 22:12:38.492038 23699 raft_consensus.cc:2229] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:38.492779 23699 raft_consensus.cc:2258] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:38.492708 22791 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:38.493382 22791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:38.494066 23698 ts_tablet_manager.cc:1905] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:38.494731 23699 ts_tablet_manager.cc:1905] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:38.500949 23698 ts_tablet_manager.cc:1918] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:38.501165 23698 log.cc:1196] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/e6618a90e1f84ca5b46223502f2b84b3
I20260811 22:12:38.501731 23698 ts_tablet_manager.cc:1939] T e6618a90e1f84ca5b46223502f2b84b3 P f76309604a1e459eaa85ff839797bd07: Deleting consensus metadata
I20260811 22:12:38.502727 23508 tablet_service.cc:1467] Processing CreateTablet for tablet c774360eddab4293b0e6fb808ecfc10a (DEFAULT_TABLE table=test-table [id=2611ca72cab7464b980516d700d858ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:38.503197 23508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c774360eddab4293b0e6fb808ecfc10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:38.503377 23699 ts_tablet_manager.cc:1918] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:38.503558 23699 log.cc:1196] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/e6618a90e1f84ca5b46223502f2b84b3
I20260811 22:12:38.503723 22768 catalog_manager.cc:4841] TS f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483): tablet e6618a90e1f84ca5b46223502f2b84b3 (table test-table [id=d06ffd62d2d44f2fa74e7cc9d489d809]) successfully deleted
I20260811 22:12:38.504191 23699 ts_tablet_manager.cc:1939] T e6618a90e1f84ca5b46223502f2b84b3 P f923495ebd9945979ae679a6409be9b9: Deleting consensus metadata
I20260811 22:12:38.506024 22768 catalog_manager.cc:4841] TS f923495ebd9945979ae679a6409be9b9 (127.10.173.130:42959): tablet e6618a90e1f84ca5b46223502f2b84b3 (table test-table [id=d06ffd62d2d44f2fa74e7cc9d489d809]) successfully deleted
I20260811 22:12:38.506745 23267 tablet_service.cc:1467] Processing CreateTablet for tablet c774360eddab4293b0e6fb808ecfc10a (DEFAULT_TABLE table=test-table [id=2611ca72cab7464b980516d700d858ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:38.507040 23021 tablet_service.cc:1467] Processing CreateTablet for tablet c774360eddab4293b0e6fb808ecfc10a (DEFAULT_TABLE table=test-table [id=2611ca72cab7464b980516d700d858ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:38.507191 23267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c774360eddab4293b0e6fb808ecfc10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:38.507640 23021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c774360eddab4293b0e6fb808ecfc10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:38.508932 23659 tablet_bootstrap.cc:492] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07: Bootstrap starting.
I20260811 22:12:38.510622 23659 tablet_bootstrap.cc:654] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:38.514951 23658 tablet_bootstrap.cc:492] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9: Bootstrap starting.
I20260811 22:12:38.515863 23660 tablet_bootstrap.cc:492] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9: Bootstrap starting.
I20260811 22:12:38.516752 23658 tablet_bootstrap.cc:654] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:38.517819 23660 tablet_bootstrap.cc:654] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:38.524868 23701 tablet_replica.cc:330] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: stopping tablet replica
I20260811 22:12:38.525607 23701 raft_consensus.cc:2229] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:38.526243 23701 raft_consensus.cc:2258] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:38.527141 23659 tablet_bootstrap.cc:492] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07: No bootstrap required, opened a new log
I20260811 22:12:38.527308 23659 ts_tablet_manager.cc:1397] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.006s
I20260811 22:12:38.527427 23660 tablet_bootstrap.cc:492] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9: No bootstrap required, opened a new log
I20260811 22:12:38.527733 23660 ts_tablet_manager.cc:1397] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260811 22:12:38.528694 23701 ts_tablet_manager.cc:1905] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:38.528967 23660 raft_consensus.cc:348] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.529233 23659 raft_consensus.cc:348] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.529501 23659 raft_consensus.cc:374] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:38.529524 23660 raft_consensus.cc:374] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:38.529613 23659 raft_consensus.cc:729] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76309604a1e459eaa85ff839797bd07, State: Initialized, Role: FOLLOWER
I20260811 22:12:38.529747 23660 raft_consensus.cc:729] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f923495ebd9945979ae679a6409be9b9, State: Initialized, Role: FOLLOWER
I20260811 22:12:38.530014 23659 consensus_queue.cc:260] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.530120 23660 consensus_queue.cc:260] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.531961 23660 ts_tablet_manager.cc:1428] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:12:38.532598 23658 tablet_bootstrap.cc:492] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9: No bootstrap required, opened a new log
I20260811 22:12:38.532786 23658 ts_tablet_manager.cc:1397] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.005s
I20260811 22:12:38.533577 23658 raft_consensus.cc:348] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.533831 23658 raft_consensus.cc:374] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:38.533960 23658 raft_consensus.cc:729] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a97c9e03e492d9380756144b70dc9, State: Initialized, Role: FOLLOWER
I20260811 22:12:38.534346 23658 consensus_queue.cc:260] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.534881 23659 ts_tablet_manager.cc:1428] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260811 22:12:38.535691 23658 ts_tablet_manager.cc:1428] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:38.536785 23701 ts_tablet_manager.cc:1918] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:38.536957 23701 log.cc:1196] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/e6618a90e1f84ca5b46223502f2b84b3
I20260811 22:12:38.537601 23701 ts_tablet_manager.cc:1939] T e6618a90e1f84ca5b46223502f2b84b3 P 506a97c9e03e492d9380756144b70dc9: Deleting consensus metadata
I20260811 22:12:38.539451 22766 catalog_manager.cc:4841] TS 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101): tablet e6618a90e1f84ca5b46223502f2b84b3 (table test-table [id=d06ffd62d2d44f2fa74e7cc9d489d809]) successfully deleted
I20260811 22:12:38.728680 23669 raft_consensus.cc:482] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:38.728940 23669 raft_consensus.cc:504] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.730749 23669 leader_election.cc:288] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101), f923495ebd9945979ae679a6409be9b9 (127.10.173.130:42959)
I20260811 22:12:38.746696 23654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.731684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:60041 (local address 127.10.173.129:38101)
0811 22:12:38.731894 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:38.731903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:38.739945 (+ 8042us) server_negotiation.cc:407] Connection header received
0811 22:12:38.740313 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:38.740320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:38.740404 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:38.740494 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:38.741862 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.742823 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.744540 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.745509 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.745681 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.745758 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:38.746195 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 22:12:38.746355 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:12:38.747064 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.731235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:38101 (local address 127.10.173.131:60041)
0811 22:12:38.739801 (+ 8566us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:38.739828 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:38.740099 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:38.741026 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:38.741034 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:38.741069 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:38.741653 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:38.741669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.742945 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.742952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:38.744380 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:38.744392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.745734 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.745743 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:38.745949 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.745992 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:12:38.746495 (+ 503us) client_negotiation.cc:240] Negotiation successful
0811 22:12:38.746649 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8396,"mutex_wait_us":62,"thread_start_us":8622,"threads_started":1}
I20260811 22:12:38.747702 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.731823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:44717 (local address 127.10.173.130:42959)
0811 22:12:38.732049 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:38.732058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:38.740298 (+ 8240us) server_negotiation.cc:407] Connection header received
0811 22:12:38.740578 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:38.740586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:38.740704 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:38.740816 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:38.741367 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.742446 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.745218 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.746513 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.746665 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.746733 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:38.747331 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 22:12:38.747483 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:12:38.747958 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c774360eddab4293b0e6fb808ecfc10a" candidate_uuid: "f76309604a1e459eaa85ff839797bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a97c9e03e492d9380756144b70dc9" is_pre_election: true
I20260811 22:12:38.748247 23054 raft_consensus.cc:2454] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76309604a1e459eaa85ff839797bd07 in term 0.
I20260811 22:12:38.748472 23709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.731379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:42959 (local address 127.10.173.131:44717)
0811 22:12:38.740163 (+ 8784us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:38.740178 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:38.740399 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:38.740843 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:38.740846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:38.740867 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:38.741204 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:38.741212 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.742597 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.742603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:38.745008 (+ 2405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:38.745027 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.746906 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.746917 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:38.747112 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.747156 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 22:12:38.747983 (+ 827us) client_negotiation.cc:240] Negotiation successful
0811 22:12:38.748114 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8688,"mutex_wait_us":2,"thread_start_us":8059,"threads_started":1}
I20260811 22:12:38.748786 23427 leader_election.cc:302] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506a97c9e03e492d9380756144b70dc9, f76309604a1e459eaa85ff839797bd07; no voters:
I20260811 22:12:38.749229 23669 raft_consensus.cc:2781] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:38.749369 23669 raft_consensus.cc:482] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:38.749481 23669 raft_consensus.cc:3037] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:38.749532 23310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c774360eddab4293b0e6fb808ecfc10a" candidate_uuid: "f76309604a1e459eaa85ff839797bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f923495ebd9945979ae679a6409be9b9" is_pre_election: true
I20260811 22:12:38.750025 23310 raft_consensus.cc:2454] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76309604a1e459eaa85ff839797bd07 in term 0.
I20260811 22:12:38.752067 23669 raft_consensus.cc:504] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.752768 23669 leader_election.cc:288] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [CANDIDATE]: Term 1 election: Requested vote from peers 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101), f923495ebd9945979ae679a6409be9b9 (127.10.173.130:42959)
I20260811 22:12:38.753149 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c774360eddab4293b0e6fb808ecfc10a" candidate_uuid: "f76309604a1e459eaa85ff839797bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a97c9e03e492d9380756144b70dc9"
I20260811 22:12:38.753410 23054 raft_consensus.cc:3037] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:38.753358 23310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c774360eddab4293b0e6fb808ecfc10a" candidate_uuid: "f76309604a1e459eaa85ff839797bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f923495ebd9945979ae679a6409be9b9"
I20260811 22:12:38.753582 23310 raft_consensus.cc:3037] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:38.756523 23054 raft_consensus.cc:2454] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76309604a1e459eaa85ff839797bd07 in term 1.
I20260811 22:12:38.757081 23427 leader_election.cc:302] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506a97c9e03e492d9380756144b70dc9, f76309604a1e459eaa85ff839797bd07; no voters:
I20260811 22:12:38.757202 23310 raft_consensus.cc:2454] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76309604a1e459eaa85ff839797bd07 in term 1.
I20260811 22:12:38.757473 23669 raft_consensus.cc:2781] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:38.761404 23669 raft_consensus.cc:686] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 1 LEADER]: Becoming Leader. State: Replica: f76309604a1e459eaa85ff839797bd07, State: Running, Role: LEADER
I20260811 22:12:38.761900 23669 consensus_queue.cc:237] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.765786 22791 catalog_manager.cc:5495] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 reported cstate change: term changed from 0 to 1, leader changed from <none> to f76309604a1e459eaa85ff839797bd07 (127.10.173.131). New cstate: current_term: 1 leader_uuid: "f76309604a1e459eaa85ff839797bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } health_report { overall_health: HEALTHY } } }
Aug 11 22:12:38 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486355, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.190@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:38.861637 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.844930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41852 (local address 127.10.173.190:46259)
0811 22:12:38.845772 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:38.845782 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:38.845990 (+ 208us) server_negotiation.cc:407] Connection header received
0811 22:12:38.846202 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:38.846207 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:38.846274 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:38.846383 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:38.847516 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.848275 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.849569 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.850004 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.850136 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.854029 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:38.854034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:38.854038 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:38.854070 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:38.857677 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:38.858724 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:38.858729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:38.858733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:38.858900 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:38.859389 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:38.859393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:38.859396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:38.859790 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:38.859934 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:38.860965 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0811 22:12:38.861223 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":431,"threads_started":1}
I20260811 22:12:38.862510 23726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.844953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46259 (local address 127.0.0.1:41852)
0811 22:12:38.845884 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:38.845901 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:38.846073 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:38.846453 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:38.846458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:38.847132 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:38.847350 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:38.847361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.848425 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.848430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:38.849417 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:38.849430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:38.850185 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:38.850192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:38.850377 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:38.851242 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:38.851263 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:38.853777 (+ 2514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:38.857845 (+ 4068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:38.857862 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:38.857870 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:38.858565 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:38.859039 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:38.859045 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:38.859048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:38.859244 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:38.859952 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:38.859962 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:38.860183 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 22:12:38.860390 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:12:38.860710 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":675,"threads_started":1}
I20260811 22:12:38.869261 22791 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:41852:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:12:38.870682 22791 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:38.878921 23024 tablet_service.cc:1467] Processing CreateTablet for tablet 0334d8ed93614557980d2f064d631762 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbdb9353416941ea95d5aed97e4c85bd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:12:38.879555 23024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0334d8ed93614557980d2f064d631762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:38.879437 23267 tablet_service.cc:1467] Processing CreateTablet for tablet 0334d8ed93614557980d2f064d631762 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbdb9353416941ea95d5aed97e4c85bd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:12:38.880023 23267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0334d8ed93614557980d2f064d631762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:38.882949 23508 tablet_service.cc:1467] Processing CreateTablet for tablet 0334d8ed93614557980d2f064d631762 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbdb9353416941ea95d5aed97e4c85bd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:12:38.883508 23508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0334d8ed93614557980d2f064d631762. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:38.887132 23658 tablet_bootstrap.cc:492] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: Bootstrap starting.
I20260811 22:12:38.888878 23660 tablet_bootstrap.cc:492] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: Bootstrap starting.
I20260811 22:12:38.888974 23658 tablet_bootstrap.cc:654] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:38.891188 23660 tablet_bootstrap.cc:654] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:38.894012 23659 tablet_bootstrap.cc:492] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: Bootstrap starting.
I20260811 22:12:38.895925 23659 tablet_bootstrap.cc:654] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:38.901755 23659 tablet_bootstrap.cc:492] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: No bootstrap required, opened a new log
I20260811 22:12:38.901957 23659 ts_tablet_manager.cc:1397] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 22:12:38.902496 23658 tablet_bootstrap.cc:492] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: No bootstrap required, opened a new log
I20260811 22:12:38.902690 23658 ts_tablet_manager.cc:1397] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.004s
I20260811 22:12:38.902882 23659 raft_consensus.cc:348] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.903141 23659 raft_consensus.cc:374] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:38.903231 23659 raft_consensus.cc:729] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76309604a1e459eaa85ff839797bd07, State: Initialized, Role: FOLLOWER
I20260811 22:12:38.903483 23658 raft_consensus.cc:348] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.903530 23659 consensus_queue.cc:260] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.903743 23658 raft_consensus.cc:374] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:38.903829 23658 raft_consensus.cc:729] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a97c9e03e492d9380756144b70dc9, State: Initialized, Role: FOLLOWER
I20260811 22:12:38.903971 23669 tablet_replica.cc:439] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } }
I20260811 22:12:38.904270 23669 tablet_replica.cc:442] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:12:38.904114 23658 consensus_queue.cc:260] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.904582 23667 tablet_replica.cc:439] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } }
I20260811 22:12:38.904925 23667 tablet_replica.cc:442] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:12:38.903679 23660 tablet_bootstrap.cc:492] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: No bootstrap required, opened a new log
I20260811 22:12:38.905582 23660 ts_tablet_manager.cc:1397] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.000s
I20260811 22:12:38.906477 23660 raft_consensus.cc:348] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.906759 23659 ts_tablet_manager.cc:1428] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:12:38.906817 23660 raft_consensus.cc:374] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:38.907012 23660 raft_consensus.cc:729] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f923495ebd9945979ae679a6409be9b9, State: Initialized, Role: FOLLOWER
I20260811 22:12:38.907280 23660 consensus_queue.cc:260] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.907934 23660 ts_tablet_manager.cc:1428] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:12:38.908021 23691 tablet_replica.cc:439] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } }
I20260811 22:12:38.908274 23658 ts_tablet_manager.cc:1428] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:12:38.908407 23691 tablet_replica.cc:442] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:12:38.987181 23669 raft_consensus.cc:482] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:38.987452 23669 raft_consensus.cc:504] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.988178 23669 leader_election.cc:288] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101), f923495ebd9945979ae679a6409be9b9 (127.10.173.130:42959)
I20260811 22:12:38.988752 23310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0334d8ed93614557980d2f064d631762" candidate_uuid: "f76309604a1e459eaa85ff839797bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f923495ebd9945979ae679a6409be9b9" is_pre_election: true
I20260811 22:12:38.988787 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0334d8ed93614557980d2f064d631762" candidate_uuid: "f76309604a1e459eaa85ff839797bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a97c9e03e492d9380756144b70dc9" is_pre_election: true
I20260811 22:12:38.989038 23310 raft_consensus.cc:2454] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76309604a1e459eaa85ff839797bd07 in term 0.
I20260811 22:12:38.989070 23054 raft_consensus.cc:2454] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76309604a1e459eaa85ff839797bd07 in term 0.
I20260811 22:12:38.989539 23427 leader_election.cc:302] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506a97c9e03e492d9380756144b70dc9, f76309604a1e459eaa85ff839797bd07; no voters:
I20260811 22:12:38.989862 23669 raft_consensus.cc:2781] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:38.989967 23669 raft_consensus.cc:482] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:38.990069 23669 raft_consensus.cc:3037] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:38.992771 23669 raft_consensus.cc:504] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:38.993270 23669 leader_election.cc:288] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [CANDIDATE]: Term 1 election: Requested vote from peers 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101), f923495ebd9945979ae679a6409be9b9 (127.10.173.130:42959)
I20260811 22:12:38.993839 23310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0334d8ed93614557980d2f064d631762" candidate_uuid: "f76309604a1e459eaa85ff839797bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f923495ebd9945979ae679a6409be9b9"
I20260811 22:12:38.993848 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0334d8ed93614557980d2f064d631762" candidate_uuid: "f76309604a1e459eaa85ff839797bd07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a97c9e03e492d9380756144b70dc9"
I20260811 22:12:38.994078 23310 raft_consensus.cc:3037] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:38.994079 23054 raft_consensus.cc:3037] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:38.996984 23054 raft_consensus.cc:2454] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76309604a1e459eaa85ff839797bd07 in term 1.
I20260811 22:12:38.996984 23310 raft_consensus.cc:2454] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76309604a1e459eaa85ff839797bd07 in term 1.
I20260811 22:12:38.997476 23429 leader_election.cc:302] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f76309604a1e459eaa85ff839797bd07, f923495ebd9945979ae679a6409be9b9; no voters:
I20260811 22:12:38.997795 23669 raft_consensus.cc:2781] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:38.997973 23669 raft_consensus.cc:686] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [term 1 LEADER]: Becoming Leader. State: Replica: f76309604a1e459eaa85ff839797bd07, State: Running, Role: LEADER
I20260811 22:12:38.998237 23669 consensus_queue.cc:237] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.000010 23713 tablet_replica.cc:439] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: TxnStatusTablet state changed. Reason: New leader f76309604a1e459eaa85ff839797bd07. Latest consensus state: current_term: 1 leader_uuid: "f76309604a1e459eaa85ff839797bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } }
I20260811 22:12:39.000428 23713 tablet_replica.cc:442] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:12:39.001088 23732 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:12:39.001344 23732 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:12:39.001454 22788 catalog_manager.cc:5495] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 reported cstate change: term changed from 0 to 1, leader changed from <none> to f76309604a1e459eaa85ff839797bd07 (127.10.173.131). New cstate: current_term: 1 leader_uuid: "f76309604a1e459eaa85ff839797bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } health_report { overall_health: HEALTHY } } }
I20260811 22:12:39.039350 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.027234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52916 (local address 127.10.173.131:39483)
0811 22:12:39.031995 (+ 4761us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:39.032001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:39.032023 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:12:39.032123 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:39.032128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:39.032278 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:39.032523 (+ 245us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:39.033031 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.034326 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.036365 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.037074 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.037207 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.037772 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:39.038588 (+ 816us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:39.038741 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:39.038863 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 22:12:39.038960 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4550,"thread_start_us":4281,"threads_started":1}
I20260811 22:12:39.039698 23726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.027232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:39483 (local address 127.0.0.1:52916)
0811 22:12:39.027710 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:39.027733 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:39.027889 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:39.032576 (+ 4687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:39.032582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:39.032596 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:12:39.032813 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:39.032826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.034504 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.034509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:39.036213 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:39.036229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.037241 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.037248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:39.037446 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.037522 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:12:39.038706 (+ 1184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:12:39.038730 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:12:39.038817 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:12:39.039222 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260811 22:12:39.044780 23054 raft_consensus.cc:1261] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Refusing update from remote peer f76309604a1e459eaa85ff839797bd07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:39.044943 23310 raft_consensus.cc:1261] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Refusing update from remote peer f76309604a1e459eaa85ff839797bd07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:39.045661 23669 consensus_queue.cc:1035] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:39.045976 23713 consensus_queue.cc:1035] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:39.055339 23691 tablet_replica.cc:439] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: TxnStatusTablet state changed. Reason: New leader f76309604a1e459eaa85ff839797bd07. Latest consensus state: current_term: 1 leader_uuid: "f76309604a1e459eaa85ff839797bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } }
I20260811 22:12:39.055665 23691 tablet_replica.cc:442] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:12:39.059113 23667 tablet_replica.cc:439] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: TxnStatusTablet state changed. Reason: New leader f76309604a1e459eaa85ff839797bd07. Latest consensus state: current_term: 1 leader_uuid: "f76309604a1e459eaa85ff839797bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } }
I20260811 22:12:39.059435 23667 tablet_replica.cc:442] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:12:39.062944 23667 tablet_replica.cc:439] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f76309604a1e459eaa85ff839797bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } }
I20260811 22:12:39.063179 23667 tablet_replica.cc:442] T 0334d8ed93614557980d2f064d631762 P 506a97c9e03e492d9380756144b70dc9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:12:39.065398 23713 tablet_replica.cc:439] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f76309604a1e459eaa85ff839797bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } }
I20260811 22:12:39.065660 23713 tablet_replica.cc:442] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:12:39.067026 23713 tablet_replica.cc:439] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f76309604a1e459eaa85ff839797bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } }
I20260811 22:12:39.067313 23713 tablet_replica.cc:442] T 0334d8ed93614557980d2f064d631762 P f76309604a1e459eaa85ff839797bd07: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:12:39.070943 23691 tablet_replica.cc:439] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f76309604a1e459eaa85ff839797bd07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } } }
I20260811 22:12:39.071209 23691 tablet_replica.cc:442] T 0334d8ed93614557980d2f064d631762 P f923495ebd9945979ae679a6409be9b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:12:39.079294 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.073283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52922 (local address 127.10.173.131:39483)
0811 22:12:39.073463 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:39.073472 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:39.073820 (+ 348us) server_negotiation.cc:407] Connection header received
0811 22:12:39.073940 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:39.073945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:39.074031 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:39.074137 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:39.074677 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.075877 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.077318 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.078007 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.078132 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.078514 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:39.078641 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:39.078752 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:39.078893 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 22:12:39.078994 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:12:39.097829 23738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.090190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:60211 (local address 127.10.173.131:39483)
0811 22:12:39.090446 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:39.090454 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:39.092852 (+ 2398us) server_negotiation.cc:407] Connection header received
0811 22:12:39.093083 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:39.093086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:39.093164 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:39.093261 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:39.093781 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.094577 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.096025 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.096425 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.096549 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.097005 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:39.097097 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:39.097193 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:39.097331 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 22:12:39.097393 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:12:39.097888 23757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.090214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:39483 (local address 127.10.173.131:60211)
0811 22:12:39.092751 (+ 2537us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:39.092766 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:39.092934 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:39.093296 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:39.093300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:39.093313 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:12:39.093625 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:39.093633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.094705 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.094713 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:39.095889 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:39.095901 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.096627 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.096640 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:39.096765 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.096821 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:12:39.097193 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:12:39.097205 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:12:39.097311 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:12:39.097434 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2381,"thread_start_us":2223,"threads_started":1}
I20260811 22:12:39.108244 23054 raft_consensus.cc:1261] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Refusing update from remote peer f76309604a1e459eaa85ff839797bd07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:39.108289 23310 raft_consensus.cc:1261] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Refusing update from remote peer f76309604a1e459eaa85ff839797bd07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:39.108948 23713 consensus_queue.cc:1035] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:39.109285 23669 consensus_queue.cc:1035] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:39.117779 23741 mvcc.cc:205] Tried to move back new op lower bound from 7317448126902308864 to 7317448125492912128. Current Snapshot: MvccSnapshot[applied={T|T < 7317448126902308864}]
I20260811 22:12:39.172173 22788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:39.172700 22788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:39.175840 22788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9: Sending DeleteTablet for 3 replicas of tablet c774360eddab4293b0e6fb808ecfc10a
I20260811 22:12:39.176738 23024 tablet_service.cc:1514] Processing DeleteTablet for tablet c774360eddab4293b0e6fb808ecfc10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:39 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:38244
I20260811 22:12:39.176824 23508 tablet_service.cc:1514] Processing DeleteTablet for tablet c774360eddab4293b0e6fb808ecfc10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:39 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:52910
I20260811 22:12:39.177244 23267 tablet_service.cc:1514] Processing DeleteTablet for tablet c774360eddab4293b0e6fb808ecfc10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:39 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:46010
I20260811 22:12:39.188730 23769 tablet_replica.cc:330] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07: stopping tablet replica
I20260811 22:12:39.189142 23769 raft_consensus.cc:2229] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:39.189889 23769 raft_consensus.cc:2258] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:39.191159 23769 ts_tablet_manager.cc:1905] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:39.195263 23767 tablet_replica.cc:330] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9: stopping tablet replica
I20260811 22:12:39.195487 23767 raft_consensus.cc:2229] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:39.195797 23767 raft_consensus.cc:2258] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:39.197013 23767 ts_tablet_manager.cc:1905] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:39.199772 23769 ts_tablet_manager.cc:1918] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:12:39.199971 23769 log.cc:1196] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/c774360eddab4293b0e6fb808ecfc10a
I20260811 22:12:39.200017 23770 tablet_replica.cc:330] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9: stopping tablet replica
I20260811 22:12:39.200259 23770 raft_consensus.cc:2229] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:39.200492 23770 raft_consensus.cc:2258] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:39.201265 23769 ts_tablet_manager.cc:1939] T c774360eddab4293b0e6fb808ecfc10a P f76309604a1e459eaa85ff839797bd07: Deleting consensus metadata
I20260811 22:12:39.201535 23770 ts_tablet_manager.cc:1905] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:39.202658 22768 catalog_manager.cc:4841] TS f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483): tablet c774360eddab4293b0e6fb808ecfc10a (table test-table [id=2611ca72cab7464b980516d700d858ac]) successfully deleted
I20260811 22:12:39.204020 23767 ts_tablet_manager.cc:1918] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:12:39.204169 23767 log.cc:1196] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/c774360eddab4293b0e6fb808ecfc10a
I20260811 22:12:39.204612 23767 ts_tablet_manager.cc:1939] T c774360eddab4293b0e6fb808ecfc10a P 506a97c9e03e492d9380756144b70dc9: Deleting consensus metadata
I20260811 22:12:39.205611 22766 catalog_manager.cc:4841] TS 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101): tablet c774360eddab4293b0e6fb808ecfc10a (table test-table [id=2611ca72cab7464b980516d700d858ac]) successfully deleted
I20260811 22:12:39.208659 23770 ts_tablet_manager.cc:1918] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:12:39.208854 23770 log.cc:1196] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/c774360eddab4293b0e6fb808ecfc10a
I20260811 22:12:39.209338 23770 ts_tablet_manager.cc:1939] T c774360eddab4293b0e6fb808ecfc10a P f923495ebd9945979ae679a6409be9b9: Deleting consensus metadata
I20260811 22:12:39.210453 22768 catalog_manager.cc:4841] TS f923495ebd9945979ae679a6409be9b9 (127.10.173.130:42959): tablet c774360eddab4293b0e6fb808ecfc10a (table test-table [id=2611ca72cab7464b980516d700d858ac]) successfully deleted
Aug 11 22:12:39 dist-test-slave-68xk krb5kdc[22710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:12:39 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:39 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:12:39.255530 23725 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:39.243868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.10.173.190:46259)
0811 22:12:39.244128 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:39.244134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:39.244467 (+ 333us) server_negotiation.cc:407] Connection header received
0811 22:12:39.244667 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:39.244671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:39.244764 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:39.244845 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:39.246641 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.247454 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.248983 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.249425 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.249571 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.254909 (+ 5338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41856: BlockingRecv error: recv got EOF from 127.0.0.1:41856 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 22:12:39 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM
I20260811 22:12:39.274523 23725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.258061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41862 (local address 127.10.173.190:46259)
0811 22:12:39.258285 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:39.258291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:39.258655 (+ 364us) server_negotiation.cc:407] Connection header received
0811 22:12:39.258830 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:39.258833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:39.258894 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:39.258974 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:39.260550 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.261341 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.263222 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.263659 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.263760 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.268459 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:39.268464 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:39.268467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:39.268503 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:39.271507 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:39.272336 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:39.272340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:39.272342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:39.272468 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:39.272926 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:39.272930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:39.272932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:39.273274 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:39.273551 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:39.273848 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 22:12:39.274105 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:12:39.278251 22788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:39.278985 22788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:39.286034 23024 tablet_service.cc:1467] Processing CreateTablet for tablet 1fcb53bbeb0a4d0f8083e808e9bd7444 (DEFAULT_TABLE table=test-table [id=7c7a2d93ca6e4161bf4a8f48c6ba1ab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:39.286175 23508 tablet_service.cc:1467] Processing CreateTablet for tablet 1fcb53bbeb0a4d0f8083e808e9bd7444 (DEFAULT_TABLE table=test-table [id=7c7a2d93ca6e4161bf4a8f48c6ba1ab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:39.286511 23024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fcb53bbeb0a4d0f8083e808e9bd7444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:39.286700 23508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fcb53bbeb0a4d0f8083e808e9bd7444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:39.290879 23267 tablet_service.cc:1467] Processing CreateTablet for tablet 1fcb53bbeb0a4d0f8083e808e9bd7444 (DEFAULT_TABLE table=test-table [id=7c7a2d93ca6e4161bf4a8f48c6ba1ab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:39.291360 23267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fcb53bbeb0a4d0f8083e808e9bd7444. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:39.291538 23659 tablet_bootstrap.cc:492] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07: Bootstrap starting.
I20260811 22:12:39.293349 23659 tablet_bootstrap.cc:654] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:39.296274 23658 tablet_bootstrap.cc:492] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9: Bootstrap starting.
I20260811 22:12:39.297104 23660 tablet_bootstrap.cc:492] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9: Bootstrap starting.
I20260811 22:12:39.297899 23658 tablet_bootstrap.cc:654] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:39.299067 23660 tablet_bootstrap.cc:654] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:39.307752 23658 tablet_bootstrap.cc:492] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9: No bootstrap required, opened a new log
I20260811 22:12:39.307933 23658 ts_tablet_manager.cc:1397] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260811 22:12:39.308136 23659 tablet_bootstrap.cc:492] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07: No bootstrap required, opened a new log
I20260811 22:12:39.308442 23659 ts_tablet_manager.cc:1397] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20260811 22:12:39.308881 23658 raft_consensus.cc:348] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.309181 23658 raft_consensus.cc:374] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:39.309302 23658 raft_consensus.cc:729] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a97c9e03e492d9380756144b70dc9, State: Initialized, Role: FOLLOWER
I20260811 22:12:39.309597 23658 consensus_queue.cc:260] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.310230 23659 raft_consensus.cc:348] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.310990 23658 ts_tablet_manager.cc:1428] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 22:12:39.311419 23659 raft_consensus.cc:374] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:39.311596 23659 raft_consensus.cc:729] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76309604a1e459eaa85ff839797bd07, State: Initialized, Role: FOLLOWER
I20260811 22:12:39.312142 23659 consensus_queue.cc:260] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.312908 23659 ts_tablet_manager.cc:1428] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:12:39.321167 23660 tablet_bootstrap.cc:492] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9: No bootstrap required, opened a new log
I20260811 22:12:39.321345 23660 ts_tablet_manager.cc:1397] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.008s
I20260811 22:12:39.322122 23660 raft_consensus.cc:348] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.322398 23660 raft_consensus.cc:374] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:39.322525 23660 raft_consensus.cc:729] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f923495ebd9945979ae679a6409be9b9, State: Initialized, Role: FOLLOWER
I20260811 22:12:39.323139 23660 consensus_queue.cc:260] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.323838 23660 ts_tablet_manager.cc:1428] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 22:12:39.512583 23691 raft_consensus.cc:482] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:39.512871 23691 raft_consensus.cc:504] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.513747 23691 leader_election.cc:288] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101), f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483)
I20260811 22:12:39.514382 23545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcb53bbeb0a4d0f8083e808e9bd7444" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76309604a1e459eaa85ff839797bd07" is_pre_election: true
I20260811 22:12:39.514429 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcb53bbeb0a4d0f8083e808e9bd7444" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a97c9e03e492d9380756144b70dc9" is_pre_election: true
I20260811 22:12:39.514727 23054 raft_consensus.cc:2454] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 0.
I20260811 22:12:39.514750 23545 raft_consensus.cc:2454] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 0.
I20260811 22:12:39.515275 23172 leader_election.cc:302] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506a97c9e03e492d9380756144b70dc9, f923495ebd9945979ae679a6409be9b9; no voters:
I20260811 22:12:39.515631 23691 raft_consensus.cc:2781] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:39.515724 23691 raft_consensus.cc:482] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:39.515838 23691 raft_consensus.cc:3037] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:39.517869 23691 raft_consensus.cc:504] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.518445 23691 leader_election.cc:288] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 election: Requested vote from peers 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101), f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483)
I20260811 22:12:39.518860 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcb53bbeb0a4d0f8083e808e9bd7444" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a97c9e03e492d9380756144b70dc9"
I20260811 22:12:39.518967 23545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fcb53bbeb0a4d0f8083e808e9bd7444" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76309604a1e459eaa85ff839797bd07"
I20260811 22:12:39.519066 23054 raft_consensus.cc:3037] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:39.519222 23545 raft_consensus.cc:3037] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:39.521665 23545 raft_consensus.cc:2454] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 1.
I20260811 22:12:39.522135 23054 raft_consensus.cc:2454] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 1.
I20260811 22:12:39.522167 23177 leader_election.cc:302] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f76309604a1e459eaa85ff839797bd07, f923495ebd9945979ae679a6409be9b9; no voters:
I20260811 22:12:39.522534 23691 raft_consensus.cc:2781] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:39.522996 23691 raft_consensus.cc:686] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 1 LEADER]: Becoming Leader. State: Replica: f923495ebd9945979ae679a6409be9b9, State: Running, Role: LEADER
I20260811 22:12:39.523327 23691 consensus_queue.cc:237] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.526469 22791 catalog_manager.cc:5495] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f923495ebd9945979ae679a6409be9b9 (127.10.173.130). New cstate: current_term: 1 leader_uuid: "f923495ebd9945979ae679a6409be9b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } health_report { overall_health: UNKNOWN } } }
I20260811 22:12:39.548889 23803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.542758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46022 (local address 127.10.173.130:42959)
0811 22:12:39.543416 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:39.543423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:39.543463 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:12:39.543540 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:39.543546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:39.543645 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:39.543758 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:39.544110 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.545299 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.546664 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.547324 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.547448 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.548034 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:39.548186 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:39.548281 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:39.548423 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 22:12:39.548509 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":342,"threads_started":1}
I20260811 22:12:39.551833 23054 raft_consensus.cc:1261] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Refusing update from remote peer f923495ebd9945979ae679a6409be9b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:39.551939 23545 raft_consensus.cc:1261] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Refusing update from remote peer f923495ebd9945979ae679a6409be9b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:39.552441 23691 consensus_queue.cc:1035] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:39.552809 23798 consensus_queue.cc:1035] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:39.565090 22785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:39.565641 22785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:39.568924 22785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9: Sending DeleteTablet for 3 replicas of tablet 1fcb53bbeb0a4d0f8083e808e9bd7444
I20260811 22:12:39.569695 23024 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fcb53bbeb0a4d0f8083e808e9bd7444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:39 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:38244
I20260811 22:12:39.570008 23767 tablet_replica.cc:330] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9: stopping tablet replica
I20260811 22:12:39.570168 23267 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fcb53bbeb0a4d0f8083e808e9bd7444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:39 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:46010
I20260811 22:12:39.570250 23767 raft_consensus.cc:2229] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:39.570518 23767 raft_consensus.cc:2258] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:39.570518 23508 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fcb53bbeb0a4d0f8083e808e9bd7444 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:39 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:52910
I20260811 22:12:39.570792 23770 tablet_replica.cc:330] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9: stopping tablet replica
I20260811 22:12:39.570994 23770 raft_consensus.cc:2229] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:39.571311 23770 raft_consensus.cc:2258] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:39.571583 23769 tablet_replica.cc:330] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07: stopping tablet replica
I20260811 22:12:39.571779 23769 raft_consensus.cc:2229] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:39.571818 23767 ts_tablet_manager.cc:1905] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:39.571988 23769 raft_consensus.cc:2258] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:39.573033 23770 ts_tablet_manager.cc:1905] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:39.576967 23767 ts_tablet_manager.cc:1918] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:39.577140 23767 log.cc:1196] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/1fcb53bbeb0a4d0f8083e808e9bd7444
I20260811 22:12:39.577524 23767 ts_tablet_manager.cc:1939] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P 506a97c9e03e492d9380756144b70dc9: Deleting consensus metadata
I20260811 22:12:39.578577 22766 catalog_manager.cc:4841] TS 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101): tablet 1fcb53bbeb0a4d0f8083e808e9bd7444 (table test-table [id=7c7a2d93ca6e4161bf4a8f48c6ba1ab0]) successfully deleted
I20260811 22:12:39.578951 22785 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:39.579573 22785 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:39.579669 23770 ts_tablet_manager.cc:1918] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:39.579824 23770 log.cc:1196] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/1fcb53bbeb0a4d0f8083e808e9bd7444
I20260811 22:12:39.580257 23770 ts_tablet_manager.cc:1939] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f923495ebd9945979ae679a6409be9b9: Deleting consensus metadata
I20260811 22:12:39.581395 22768 catalog_manager.cc:4841] TS f923495ebd9945979ae679a6409be9b9 (127.10.173.130:42959): tablet 1fcb53bbeb0a4d0f8083e808e9bd7444 (table test-table [id=7c7a2d93ca6e4161bf4a8f48c6ba1ab0]) successfully deleted
I20260811 22:12:39.581899 23769 ts_tablet_manager.cc:1905] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:39.587325 23024 tablet_service.cc:1467] Processing CreateTablet for tablet e5c66285d6c94d56aaaa5c2912c9efbd (DEFAULT_TABLE table=test-table [id=eb5b729dbf66468483b4f8158cba13e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:39.587814 23024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c66285d6c94d56aaaa5c2912c9efbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:39.587802 23508 tablet_service.cc:1467] Processing CreateTablet for tablet e5c66285d6c94d56aaaa5c2912c9efbd (DEFAULT_TABLE table=test-table [id=eb5b729dbf66468483b4f8158cba13e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:39.588253 23508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c66285d6c94d56aaaa5c2912c9efbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:39.593868 23659 tablet_bootstrap.cc:492] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07: Bootstrap starting.
I20260811 22:12:39.594342 23267 tablet_service.cc:1467] Processing CreateTablet for tablet e5c66285d6c94d56aaaa5c2912c9efbd (DEFAULT_TABLE table=test-table [id=eb5b729dbf66468483b4f8158cba13e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:39.594848 23267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5c66285d6c94d56aaaa5c2912c9efbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:39.595737 23659 tablet_bootstrap.cc:654] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:39.597657 23769 ts_tablet_manager.cc:1918] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:39.597910 23769 log.cc:1196] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/1fcb53bbeb0a4d0f8083e808e9bd7444
I20260811 22:12:39.598842 23769 ts_tablet_manager.cc:1939] T 1fcb53bbeb0a4d0f8083e808e9bd7444 P f76309604a1e459eaa85ff839797bd07: Deleting consensus metadata
I20260811 22:12:39.600310 22768 catalog_manager.cc:4841] TS f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483): tablet 1fcb53bbeb0a4d0f8083e808e9bd7444 (table test-table [id=7c7a2d93ca6e4161bf4a8f48c6ba1ab0]) successfully deleted
I20260811 22:12:39.601521 23658 tablet_bootstrap.cc:492] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9: Bootstrap starting.
I20260811 22:12:39.601643 23660 tablet_bootstrap.cc:492] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9: Bootstrap starting.
I20260811 22:12:39.603389 23658 tablet_bootstrap.cc:654] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:39.603669 23660 tablet_bootstrap.cc:654] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:39.610947 23659 tablet_bootstrap.cc:492] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07: No bootstrap required, opened a new log
I20260811 22:12:39.611150 23659 ts_tablet_manager.cc:1397] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.007s
I20260811 22:12:39.611220 23658 tablet_bootstrap.cc:492] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9: No bootstrap required, opened a new log
I20260811 22:12:39.611397 23658 ts_tablet_manager.cc:1397] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 22:12:39.612040 23659 raft_consensus.cc:348] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.612177 23658 raft_consensus.cc:348] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.612334 23659 raft_consensus.cc:374] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:39.612428 23658 raft_consensus.cc:374] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:39.612437 23659 raft_consensus.cc:729] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76309604a1e459eaa85ff839797bd07, State: Initialized, Role: FOLLOWER
I20260811 22:12:39.612504 23658 raft_consensus.cc:729] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 506a97c9e03e492d9380756144b70dc9, State: Initialized, Role: FOLLOWER
I20260811 22:12:39.612730 23658 consensus_queue.cc:260] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.612732 23659 consensus_queue.cc:260] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.613245 23658 ts_tablet_manager.cc:1428] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260811 22:12:39.614077 23659 ts_tablet_manager.cc:1428] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 22:12:39.614944 23660 tablet_bootstrap.cc:492] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9: No bootstrap required, opened a new log
I20260811 22:12:39.615136 23660 ts_tablet_manager.cc:1397] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.004s
I20260811 22:12:39.615880 23660 raft_consensus.cc:348] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.616143 23660 raft_consensus.cc:374] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:39.616226 23660 raft_consensus.cc:729] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f923495ebd9945979ae679a6409be9b9, State: Initialized, Role: FOLLOWER
I20260811 22:12:39.616457 23660 consensus_queue.cc:260] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.617466 23660 ts_tablet_manager.cc:1428] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:12:39.647553 23798 raft_consensus.cc:482] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:39.647795 23798 raft_consensus.cc:504] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.648478 23798 leader_election.cc:288] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101), f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483)
I20260811 22:12:39.649057 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c66285d6c94d56aaaa5c2912c9efbd" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a97c9e03e492d9380756144b70dc9" is_pre_election: true
I20260811 22:12:39.649178 23545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c66285d6c94d56aaaa5c2912c9efbd" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76309604a1e459eaa85ff839797bd07" is_pre_election: true
I20260811 22:12:39.649300 23054 raft_consensus.cc:2454] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 0.
I20260811 22:12:39.649464 23545 raft_consensus.cc:2454] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 0.
I20260811 22:12:39.649771 23172 leader_election.cc:302] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506a97c9e03e492d9380756144b70dc9, f923495ebd9945979ae679a6409be9b9; no voters:
I20260811 22:12:39.650045 23798 raft_consensus.cc:2781] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:39.650151 23798 raft_consensus.cc:482] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:39.650298 23798 raft_consensus.cc:3037] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:39.652411 23798 raft_consensus.cc:504] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.653009 23798 leader_election.cc:288] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 election: Requested vote from peers 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101), f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483)
I20260811 22:12:39.653458 23054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c66285d6c94d56aaaa5c2912c9efbd" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "506a97c9e03e492d9380756144b70dc9"
I20260811 22:12:39.653553 23545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5c66285d6c94d56aaaa5c2912c9efbd" candidate_uuid: "f923495ebd9945979ae679a6409be9b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f76309604a1e459eaa85ff839797bd07"
I20260811 22:12:39.653676 23054 raft_consensus.cc:3037] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:39.653785 23545 raft_consensus.cc:3037] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:39.656818 23054 raft_consensus.cc:2454] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 1.
I20260811 22:12:39.657166 23545 raft_consensus.cc:2454] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f923495ebd9945979ae679a6409be9b9 in term 1.
I20260811 22:12:39.657303 23172 leader_election.cc:302] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 506a97c9e03e492d9380756144b70dc9, f923495ebd9945979ae679a6409be9b9; no voters:
I20260811 22:12:39.657580 23798 raft_consensus.cc:2781] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:39.657757 23798 raft_consensus.cc:686] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 1 LEADER]: Becoming Leader. State: Replica: f923495ebd9945979ae679a6409be9b9, State: Running, Role: LEADER
I20260811 22:12:39.658016 23798 consensus_queue.cc:237] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } }
I20260811 22:12:39.660831 22792 catalog_manager.cc:5495] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f923495ebd9945979ae679a6409be9b9 (127.10.173.130). New cstate: current_term: 1 leader_uuid: "f923495ebd9945979ae679a6409be9b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f923495ebd9945979ae679a6409be9b9" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 } health_report { overall_health: UNKNOWN } } }
I20260811 22:12:39.712670 23820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.697787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:40647 (local address 127.10.173.131:39483)
0811 22:12:39.698346 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:39.698351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:39.707038 (+ 8687us) server_negotiation.cc:407] Connection header received
0811 22:12:39.707283 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:39.707287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:39.707354 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:39.707577 (+ 223us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:39.708109 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.709134 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.710745 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.711202 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.711366 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.711816 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:39.711927 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:39.712026 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:39.712193 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 22:12:39.712283 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":284,"threads_started":1}
I20260811 22:12:39.712707 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.697769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:39483 (local address 127.10.173.130:40647)
0811 22:12:39.706914 (+ 9145us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:39.706932 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:39.707126 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:39.707591 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:39.707596 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:39.707608 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:12:39.707957 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:39.707966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.709274 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.709283 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:39.710595 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:39.710608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.711395 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.711402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:39.711540 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.711602 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:12:39.712030 (+ 428us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:12:39.712045 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 22:12:39.712149 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:12:39.712293 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8974,"thread_start_us":8853,"threads_started":1}
I20260811 22:12:39.724356 23054 raft_consensus.cc:1261] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Refusing update from remote peer f923495ebd9945979ae679a6409be9b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:39.725103 23691 consensus_queue.cc:1035] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "506a97c9e03e492d9380756144b70dc9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 38101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:39.735809 23545 raft_consensus.cc:1261] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Refusing update from remote peer f923495ebd9945979ae679a6409be9b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:39.736641 23691 consensus_queue.cc:1035] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f76309604a1e459eaa85ff839797bd07" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:39.774871 23803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.767783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:48619 (local address 127.10.173.130:42959)
0811 22:12:39.767951 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:39.767960 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:39.768445 (+ 485us) server_negotiation.cc:407] Connection header received
0811 22:12:39.768682 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:39.768688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:39.768771 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:39.768866 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:39.769424 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.770544 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.772718 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.773430 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.773563 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.774054 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:39.774212 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:39.774310 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:39.774481 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:12:39.774575 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 22:12:39.775048 23829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.767717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:42959 (local address 127.10.173.131:48619)
0811 22:12:39.768339 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:39.768355 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:39.768550 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:39.768886 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:39.768892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:39.768904 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:12:39.769212 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:39.769229 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.770721 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.770727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:39.772515 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:39.772535 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.773626 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.773633 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:39.773816 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.773873 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:12:39.774333 (+ 460us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:12:39.774345 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:12:39.774464 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 22:12:39.774617 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":286,"threads_started":1}
I20260811 22:12:39.811901 22785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:39.812188 22785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:39.817237 22785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ffa02a054cf41fdbe1ee61d69ec3ff9: Sending DeleteTablet for 3 replicas of tablet e5c66285d6c94d56aaaa5c2912c9efbd
I20260811 22:12:39.818177 23267 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c66285d6c94d56aaaa5c2912c9efbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:39 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:46010
I20260811 22:12:39.818532 23770 tablet_replica.cc:330] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9: stopping tablet replica
I20260811 22:12:39.818511 23024 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c66285d6c94d56aaaa5c2912c9efbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:39 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:38244
I20260811 22:12:39.818730 23770 raft_consensus.cc:2229] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:39.818817 23767 tablet_replica.cc:330] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9: stopping tablet replica
I20260811 22:12:39.818753 23508 tablet_service.cc:1514] Processing DeleteTablet for tablet e5c66285d6c94d56aaaa5c2912c9efbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:39 UTC) from {username='oryx', principal='oryx/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:52910
I20260811 22:12:39.818981 23767 raft_consensus.cc:2229] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:39.819118 23770 raft_consensus.cc:2258] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:39.819227 23767 raft_consensus.cc:2258] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:39.819499 23769 tablet_replica.cc:330] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07: stopping tablet replica
I20260811 22:12:39.819720 23769 raft_consensus.cc:2229] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:39.819964 23769 raft_consensus.cc:2258] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:39.820551 23767 ts_tablet_manager.cc:1905] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:39.820705 23770 ts_tablet_manager.cc:1905] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:39.827468 23769 ts_tablet_manager.cc:1905] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:39.827503 23767 ts_tablet_manager.cc:1918] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:12:39.827637 23767 log.cc:1196] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/e5c66285d6c94d56aaaa5c2912c9efbd
I20260811 22:12:39.828116 23767 ts_tablet_manager.cc:1939] T e5c66285d6c94d56aaaa5c2912c9efbd P 506a97c9e03e492d9380756144b70dc9: Deleting consensus metadata
I20260811 22:12:39.829326 22766 catalog_manager.cc:4841] TS 506a97c9e03e492d9380756144b70dc9 (127.10.173.129:38101): tablet e5c66285d6c94d56aaaa5c2912c9efbd (table test-table [id=eb5b729dbf66468483b4f8158cba13e7]) successfully deleted
I20260811 22:12:39.829694 23770 ts_tablet_manager.cc:1918] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:12:39.829857 23770 log.cc:1196] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/e5c66285d6c94d56aaaa5c2912c9efbd
I20260811 22:12:39.830341 23770 ts_tablet_manager.cc:1939] T e5c66285d6c94d56aaaa5c2912c9efbd P f923495ebd9945979ae679a6409be9b9: Deleting consensus metadata
I20260811 22:12:39.831527 22768 catalog_manager.cc:4841] TS f923495ebd9945979ae679a6409be9b9 (127.10.173.130:42959): tablet e5c66285d6c94d56aaaa5c2912c9efbd (table test-table [id=eb5b729dbf66468483b4f8158cba13e7]) successfully deleted
I20260811 22:12:39.832630 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 22902
I20260811 22:12:39.835690 23769 ts_tablet_manager.cc:1918] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:12:39.835887 23769 log.cc:1196] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/e5c66285d6c94d56aaaa5c2912c9efbd
I20260811 22:12:39.836431 23769 ts_tablet_manager.cc:1939] T e5c66285d6c94d56aaaa5c2912c9efbd P f76309604a1e459eaa85ff839797bd07: Deleting consensus metadata
I20260811 22:12:39.837857 22768 catalog_manager.cc:4841] TS f76309604a1e459eaa85ff839797bd07 (127.10.173.131:39483): tablet e5c66285d6c94d56aaaa5c2912c9efbd (table test-table [id=eb5b729dbf66468483b4f8158cba13e7]) successfully deleted
Aug 11 22:12:39 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.129@KRBTEST.COM
I20260811 22:12:39.873813 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.835173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38248 (local address 127.10.173.129:38101)
0811 22:12:39.850651 (+ 15478us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:39.850661 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:39.850991 (+ 330us) server_negotiation.cc:407] Connection header received
0811 22:12:39.851237 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:39.851245 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:39.851364 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:39.851519 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:39.853602 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.854368 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.856131 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:39.856803 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:39.856920 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:39.862455 (+ 5535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:39.862479 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:39.862503 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:39.862622 (+ 119us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:39.870164 (+ 7542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:39.871104 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:39.871115 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:39.871120 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:39.871293 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:39.871731 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:39.871738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:39.871744 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:39.872289 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:39.872660 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:39.873017 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 22:12:39.873373 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15120,"thread_start_us":15293,"threads_started":1}
I20260811 22:12:39.875344 23118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:40.006711 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 22902
I20260811 22:12:40.035058 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 23140
Aug 11 22:12:40 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.130@KRBTEST.COM
I20260811 22:12:40.052561 23803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:40.035915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.10.173.130:42959)
0811 22:12:40.036091 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:40.036096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:40.036164 (+ 68us) server_negotiation.cc:407] Connection header received
0811 22:12:40.036339 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:40.036343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:40.036424 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:40.036508 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:40.038155 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:40.038916 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:40.040794 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:40.041231 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:40.041339 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:40.046294 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:40.046309 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:40.046328 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:40.046369 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:40.049458 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:40.050183 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:40.050190 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:40.050192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:40.050349 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:40.050876 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:40.050884 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:40.050888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:40.051307 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:40.051619 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:40.051991 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 22:12:40.052216 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:12:40.053272 23384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:40.186045 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 23140
I20260811 22:12:40.220170 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 23397
Aug 11 22:12:40 dist-test-slave-68xk krb5kdc[22710](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.173.131@KRBTEST.COM
I20260811 22:12:40.236227 23840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:40.221140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.10.173.131:39483)
0811 22:12:40.221664 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:40.221668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:40.221682 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:12:40.221751 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:40.221755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:40.221821 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:40.221943 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:40.223359 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:40.224049 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:40.225334 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:40.225756 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:40.225894 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:40.230446 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:40.230466 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:40.230491 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:40.230529 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:40.233455 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:40.234087 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:40.234103 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:40.234106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:40.234272 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:40.234696 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:40.234701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:40.234704 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:40.235037 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:40.235353 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:40.235667 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 22:12:40.235910 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":263,"threads_started":1}
I20260811 22:12:40.236893 23612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:40.388023 23843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:40.386459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:42959 (local address 127.10.173.131:51263)
0811 22:12:40.387728 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:40.387834 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.130:42959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1172,"thread_start_us":1099,"threads_started":1}
I20260811 22:12:40.388370 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 23397
I20260811 22:12:40.422392 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 22728
I20260811 22:12:40.423744 22853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:40.541978 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 22728
2026-08-11T22:12:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6627 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:40 dist-test-slave-68xk krb5kdc[23849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:40 dist-test-slave-68xk krb5kdc[23849](info): set up 2 sockets
Aug 11 22:12:40 dist-test-slave-68xk krb5kdc[23849](info): commencing operation
krb5kdc: starting...
W20260811 22:12:41.899952 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:12:42 dist-test-slave-68xk krb5kdc[23849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:12:42.171023 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:33875
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:33875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:42.511523 23877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:42.511884 23877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:42.512058 23877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:42.519466 23877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:42.519613 23877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:42.519748 23877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:42.519862 23877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:42.519948 23877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:42.529898 23877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:33875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:33875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:42.531241 23877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:42.532549 23877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:42.544654 23887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:42.544772 23885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:42.544708 23884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:42.545919 23877 server_base.cc:1034] running on GCE node
I20260811 22:12:42.546774 23877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:42.548064 23877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:42.549279 23877 hybrid_clock.cc:648] HybridClock initialized: now 1786486362549232 us; error 27 us; skew 500 ppm
I20260811 22:12:42.549619 23877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T22:12:42Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2075 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:42 dist-test-slave-68xk krb5kdc[23896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:42 dist-test-slave-68xk krb5kdc[23896](info): set up 2 sockets
Aug 11 22:12:42 dist-test-slave-68xk krb5kdc[23896](info): commencing operation
krb5kdc: starting...
W20260811 22:12:43.973515 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:12:44 dist-test-slave-68xk krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:12:44.251488 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:45765
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:45765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:44.589881 23918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:44.590194 23918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:44.590358 23918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:44.597728 23918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:44.597842 23918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:44.597934 23918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:44.598023 23918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:44.598093 23918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:44.608350 23918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:45765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:45765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:44.609753 23918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:44.611115 23918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:44.623459 23931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:44.623637 23927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:44.623567 23929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:44.624408 23918 server_base.cc:1034] running on GCE node
I20260811 22:12:44.625257 23918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:44.626691 23918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:44.627926 23918 hybrid_clock.cc:648] HybridClock initialized: now 1786486364627863 us; error 55 us; skew 500 ppm
I20260811 22:12:44.628325 23918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:44 dist-test-slave-68xk krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486364, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:44.631563 23918 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:12:44.633054 23918 webserver.cc:460] Webserver started at http://127.10.173.190:36461/ using document root <none> and password file <none>
I20260811 22:12:44.633770 23918 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:44.633913 23918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:44.634248 23918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:44.637255 23918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "5f9d189f72684677a30d26fbd61bcb2c"
format_stamp: "Formatted at 2026-08-11 22:12:44 on dist-test-slave-68xk"
server_key: "062d9451eb58920632d7e8d54bf4c0f0"
server_key_iv: "6d96e675e14249652bd5e47758defa09"
server_key_version: "encryptionkey@0"
I20260811 22:12:44.638116 23918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "5f9d189f72684677a30d26fbd61bcb2c"
format_stamp: "Formatted at 2026-08-11 22:12:44 on dist-test-slave-68xk"
server_key: "062d9451eb58920632d7e8d54bf4c0f0"
server_key_iv: "6d96e675e14249652bd5e47758defa09"
server_key_version: "encryptionkey@0"
I20260811 22:12:44.643431 23918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:12:44.646943 23943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:44.648367 23918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:12:44.648556 23918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "5f9d189f72684677a30d26fbd61bcb2c"
format_stamp: "Formatted at 2026-08-11 22:12:44 on dist-test-slave-68xk"
server_key: "062d9451eb58920632d7e8d54bf4c0f0"
server_key_iv: "6d96e675e14249652bd5e47758defa09"
server_key_version: "encryptionkey@0"
I20260811 22:12:44.648768 23918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:44.668292 23918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:44.673053 23918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:44.673434 23918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:44.693176 23918 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:45765
I20260811 22:12:44.693183 24027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:45765 every 8 connection(s)
I20260811 22:12:44.694720 23918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:44.698684 24028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:44.699316 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 23918
I20260811 22:12:44.699582 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:12:44.700268 10934 external_mini_cluster.cc:1428] Setting key 2c07be7bc172b82c18fdc2ff61deeada
I20260811 22:12:44.713541 24028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c: Bootstrap starting.
Aug 11 22:12:44 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:44.716943 24028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:44.718369 24028 log.cc:824] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:44.721941 24028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c: No bootstrap required, opened a new log
I20260811 22:12:44.726060 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:44.702504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55268 (local address 127.10.173.190:45765)
0811 22:12:44.703351 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:44.703366 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:44.703418 (+ 52us) server_negotiation.cc:407] Connection header received
0811 22:12:44.705622 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:44.705652 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:44.706172 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:44.706663 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:44.708102 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:44.710147 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:44.711534 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:44.712595 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:44.712792 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:44.717188 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:44.717200 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:44.717219 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:44.717292 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:44.721361 (+ 4069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:44.722088 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:44.722096 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:44.722104 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:44.722320 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:44.722779 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:44.722784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:44.722788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:44.723494 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:44.723816 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:44.724199 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 22:12:44.724668 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":326,"threads_started":1}
I20260811 22:12:44.731675 24028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } }
I20260811 22:12:44.732035 24028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:44.732110 24028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9d189f72684677a30d26fbd61bcb2c, State: Initialized, Role: FOLLOWER
I20260811 22:12:44.732682 24028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } }
I20260811 22:12:44.732954 24028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:44.733114 24028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:44.733299 24028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:44.735658 24028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } }
I20260811 22:12:44.736204 24028 leader_election.cc:302] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f9d189f72684677a30d26fbd61bcb2c; no voters:
I20260811 22:12:44.736826 24028 leader_election.cc:288] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:44.737003 24038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:44.737793 24038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 1 LEADER]: Becoming Leader. State: Replica: 5f9d189f72684677a30d26fbd61bcb2c, State: Running, Role: LEADER
I20260811 22:12:44.738471 24038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } }
I20260811 22:12:44.739308 24028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:44.741966 24042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f9d189f72684677a30d26fbd61bcb2c. Latest consensus state: current_term: 1 leader_uuid: "5f9d189f72684677a30d26fbd61bcb2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } } }
I20260811 22:12:44.741920 24041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f9d189f72684677a30d26fbd61bcb2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } } }
I20260811 22:12:44.742398 24042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:44.742446 24041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:44.744273 24050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:44.749508 24050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:44.758971 24050 catalog_manager.cc:1347] Generated new cluster ID: 7446596edc524b3f90381aeb34c3db6e
I20260811 22:12:44.759094 24050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:44.782254 24050 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:44.784199 24050 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:44.790573 24050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c: Generated new TSK 0
I20260811 22:12:44.791447 24050 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:12:44.859022 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45765
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:45.217767 24072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:45.218063 24072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:45.218196 24072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:45.225945 24072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:45.226060 24072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:45.226257 24072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:12:45.236546 24072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:45.237921 24072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:45.239394 24072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:45.253693 24078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:45.253691 24079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:45.254102 24082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:45.254999 24072 server_base.cc:1034] running on GCE node
I20260811 22:12:45.255734 24072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:45.256742 24072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:45.257987 24072 hybrid_clock.cc:648] HybridClock initialized: now 1786486365257929 us; error 46 us; skew 500 ppm
I20260811 22:12:45.258314 24072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:45 dist-test-slave-68xk krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486365, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:45.261957 24072 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:12:45.263607 24072 webserver.cc:460] Webserver started at http://127.10.173.129:41941/ using document root <none> and password file <none>
I20260811 22:12:45.264422 24072 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:45.264557 24072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:45.264851 24072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:45.268276 24072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "170954bd1a5f43e58dea5c7b2ae558c2"
format_stamp: "Formatted at 2026-08-11 22:12:45 on dist-test-slave-68xk"
server_key: "25436719adcc34d42cafb8d746a1781b"
server_key_iv: "b34f34f02b2687e1bccecc89525e05b3"
server_key_version: "encryptionkey@0"
I20260811 22:12:45.269173 24072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "170954bd1a5f43e58dea5c7b2ae558c2"
format_stamp: "Formatted at 2026-08-11 22:12:45 on dist-test-slave-68xk"
server_key: "25436719adcc34d42cafb8d746a1781b"
server_key_iv: "b34f34f02b2687e1bccecc89525e05b3"
server_key_version: "encryptionkey@0"
I20260811 22:12:45.276002 24072 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260811 22:12:45.280113 24094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:45.281714 24072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:45.281915 24072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "170954bd1a5f43e58dea5c7b2ae558c2"
format_stamp: "Formatted at 2026-08-11 22:12:45 on dist-test-slave-68xk"
server_key: "25436719adcc34d42cafb8d746a1781b"
server_key_iv: "b34f34f02b2687e1bccecc89525e05b3"
server_key_version: "encryptionkey@0"
I20260811 22:12:45.282112 24072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:45.309360 24072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:45.314255 24072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:45.314635 24072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:45.315676 24072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:45.317162 24072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:45.317243 24072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:45.317338 24072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:45.317392 24072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:45.355954 24072 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:35143
I20260811 22:12:45.356055 24276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:35143 every 8 connection(s)
I20260811 22:12:45.357405 24072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:45.358139 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 24072
I20260811 22:12:45.358399 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:12:45.359292 10934 external_mini_cluster.cc:1428] Setting key 0f694d3387e61efe068592fd6c8b5231
Aug 11 22:12:45 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486365, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:45.378930 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:45.360602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:60237 (local address 127.10.173.190:45765)
0811 22:12:45.361134 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:45.361139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:45.362311 (+ 1172us) server_negotiation.cc:407] Connection header received
0811 22:12:45.364211 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:45.364221 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:45.364326 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:45.364458 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:45.365993 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:45.367320 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:45.368916 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:45.369339 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:45.369484 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:45.373110 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:45.373115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:45.373118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:45.373152 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:45.375836 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:45.376568 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:45.376574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:45.376577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:45.376704 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:45.377142 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:45.377147 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:45.377151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:45.377435 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:45.377596 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:45.378289 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 22:12:45.378544 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":295,"threads_started":1}
I20260811 22:12:45.380041 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:45.361245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:60237)
0811 22:12:45.362111 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:45.362167 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:45.363936 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:45.364660 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:45.364668 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:45.365198 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:45.365808 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:45.365826 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:45.367484 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:45.367488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:45.368752 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:45.368776 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:45.369577 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:45.369585 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:45.369838 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:45.370570 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:45.370592 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:45.372881 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:45.375985 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:45.375994 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:45.376011 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:45.376442 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:45.376824 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:45.376829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:45.376833 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:45.377025 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:45.377578 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:45.377586 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:45.377914 (+ 328us) client_negotiation.cc:769] Sending connection context
0811 22:12:45.378188 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 22:12:45.378749 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":356,"threads_started":1}
I20260811 22:12:45.381695 24277 heartbeater.cc:347] Connected to a master server at 127.10.173.190:45765
I20260811 22:12:45.382100 24277 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:45.383123 24277 heartbeater.cc:511] Master 127.10.173.190:45765 requested a full tablet report, sending...
I20260811 22:12:45.385565 23968 ts_manager.cc:194] Registered new tserver with Master: 170954bd1a5f43e58dea5c7b2ae558c2 (127.10.173.129:35143)
I20260811 22:12:45.387390 23968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:60237
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:12:45.427019 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45765
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:45.795768 24286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:45.796078 24286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:45.796253 24286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:45.803738 24286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:45.803872 24286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:45.804271 24286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:12:45.814893 24286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:45.816336 24286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:45.817787 24286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:45.831648 24294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:45.831722 24292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:45.832350 24291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:45.833729 24286 server_base.cc:1034] running on GCE node
I20260811 22:12:45.834455 24286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:45.835492 24286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:45.836712 24286 hybrid_clock.cc:648] HybridClock initialized: now 1786486365836664 us; error 36 us; skew 500 ppm
I20260811 22:12:45.837050 24286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:45 dist-test-slave-68xk krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486365, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:45.840675 24286 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:12:45.842294 24286 webserver.cc:460] Webserver started at http://127.10.173.130:45913/ using document root <none> and password file <none>
I20260811 22:12:45.843078 24286 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:45.843184 24286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:45.843546 24286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:45.846834 24286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "f3958ca56ef94d278901beb3de22c5f3"
format_stamp: "Formatted at 2026-08-11 22:12:45 on dist-test-slave-68xk"
server_key: "7233170248349aebb3c820fcf00c7be9"
server_key_iv: "058a7ed0e08e20965c83f3c4ae2c6e4f"
server_key_version: "encryptionkey@0"
I20260811 22:12:45.847679 24286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "f3958ca56ef94d278901beb3de22c5f3"
format_stamp: "Formatted at 2026-08-11 22:12:45 on dist-test-slave-68xk"
server_key: "7233170248349aebb3c820fcf00c7be9"
server_key_iv: "058a7ed0e08e20965c83f3c4ae2c6e4f"
server_key_version: "encryptionkey@0"
I20260811 22:12:45.853516 24286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:12:45.857376 24309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:45.859138 24286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:12:45.859355 24286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "f3958ca56ef94d278901beb3de22c5f3"
format_stamp: "Formatted at 2026-08-11 22:12:45 on dist-test-slave-68xk"
server_key: "7233170248349aebb3c820fcf00c7be9"
server_key_iv: "058a7ed0e08e20965c83f3c4ae2c6e4f"
server_key_version: "encryptionkey@0"
I20260811 22:12:45.859560 24286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:45.890687 24286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:45.895629 24286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:45.895958 24286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:45.896991 24286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:45.898479 24286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:45.898559 24286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:45.898700 24286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:45.898764 24286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:45.934345 24286 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:38913
I20260811 22:12:45.934489 24481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:38913 every 8 connection(s)
I20260811 22:12:45.935750 24286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:45.938707 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 24286
I20260811 22:12:45.938951 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:12:45.939517 10934 external_mini_cluster.cc:1428] Setting key 58193d28621eb0c199e20ad6da2651c3
Aug 11 22:12:45 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486365, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:45.958945 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:45.938533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:38065 (local address 127.10.173.190:45765)
0811 22:12:45.939048 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:45.939055 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:45.940106 (+ 1051us) server_negotiation.cc:407] Connection header received
0811 22:12:45.942331 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:45.942339 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:45.942467 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:45.942589 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:45.944285 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:45.945532 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:45.947293 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:45.948042 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:45.948201 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:45.952019 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:45.952029 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:45.952036 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:45.952087 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:45.955378 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:45.956408 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:45.956413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:45.956415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:45.956548 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:45.957086 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:45.957089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:45.957091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:45.957364 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:45.957475 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:45.958472 (+ 997us) server_negotiation.cc:299] Negotiation successful
0811 22:12:45.958653 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":253,"threads_started":1}
I20260811 22:12:45.960631 24487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:45.939126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:38065)
0811 22:12:45.939934 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:45.939991 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:45.942055 (+ 2064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:45.942807 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:45.942815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:45.943380 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:45.944022 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:45.944041 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:45.945723 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:45.945727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:45.947138 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:45.947151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:45.948323 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:45.948329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:45.948534 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:45.949283 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:45.949306 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:45.951753 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:45.955535 (+ 3782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:45.955548 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:45.955569 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:45.956272 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:45.956674 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:45.956681 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:45.956688 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:45.956966 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:45.957495 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:45.957506 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:45.958001 (+ 495us) client_negotiation.cc:769] Sending connection context
0811 22:12:45.958351 (+ 350us) client_negotiation.cc:240] Negotiation successful
0811 22:12:45.959027 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":343,"threads_started":1}
I20260811 22:12:45.962352 24482 heartbeater.cc:347] Connected to a master server at 127.10.173.190:45765
I20260811 22:12:45.962762 24482 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:45.963737 24482 heartbeater.cc:511] Master 127.10.173.190:45765 requested a full tablet report, sending...
I20260811 22:12:45.965443 23968 ts_manager.cc:194] Registered new tserver with Master: f3958ca56ef94d278901beb3de22c5f3 (127.10.173.130:38913)
I20260811 22:12:45.966825 23968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:38065
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:12:46.006821 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45765
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:46.358951 24492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:46.359306 24492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:46.359472 24492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:46.367136 24492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:46.367295 24492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:46.367462 24492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:12:46.377355 24492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:46.378736 24492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:46.380043 24492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:12:46.390654 24277 heartbeater.cc:503] Master 127.10.173.190:45765 was elected leader, sending a full tablet report...
W20260811 22:12:46.393985 24497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:46.393970 24498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:46.395080 24503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:46.395910 24492 server_base.cc:1034] running on GCE node
I20260811 22:12:46.396585 24492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:46.397636 24492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:46.398886 24492 hybrid_clock.cc:648] HybridClock initialized: now 1786486366398832 us; error 37 us; skew 500 ppm
I20260811 22:12:46.399243 24492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:46 dist-test-slave-68xk krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486366, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:46.402868 24492 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:12:46.404448 24492 webserver.cc:460] Webserver started at http://127.10.173.131:34601/ using document root <none> and password file <none>
I20260811 22:12:46.405261 24492 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:46.405426 24492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:46.405788 24492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:46.409022 24492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "76291fdd94fb4cf88581599dceb98e3c"
format_stamp: "Formatted at 2026-08-11 22:12:46 on dist-test-slave-68xk"
server_key: "146ae1ed8e2ff1e398a1b305eb924bca"
server_key_iv: "8c3ef1cfed1621fb4bb074c74021c610"
server_key_version: "encryptionkey@0"
I20260811 22:12:46.409899 24492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "76291fdd94fb4cf88581599dceb98e3c"
format_stamp: "Formatted at 2026-08-11 22:12:46 on dist-test-slave-68xk"
server_key: "146ae1ed8e2ff1e398a1b305eb924bca"
server_key_iv: "8c3ef1cfed1621fb4bb074c74021c610"
server_key_version: "encryptionkey@0"
I20260811 22:12:46.415778 24492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 22:12:46.419723 24515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:46.421293 24492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:12:46.421715 24492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "76291fdd94fb4cf88581599dceb98e3c"
format_stamp: "Formatted at 2026-08-11 22:12:46 on dist-test-slave-68xk"
server_key: "146ae1ed8e2ff1e398a1b305eb924bca"
server_key_iv: "8c3ef1cfed1621fb4bb074c74021c610"
server_key_version: "encryptionkey@0"
I20260811 22:12:46.421942 24492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:46.462561 24492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:46.467435 24492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:46.467760 24492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:46.468863 24492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:46.470335 24492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:46.470458 24492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:46.470623 24492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:46.470696 24492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:46.507421 24492 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:35673
I20260811 22:12:46.507519 24705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:35673 every 8 connection(s)
I20260811 22:12:46.508744 24492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:46.516896 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 24492
I20260811 22:12:46.517143 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:46.517661 10934 external_mini_cluster.cc:1428] Setting key 3e40cbc7a405dbc9b28b992fc1b861e0
Aug 11 22:12:46 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486366, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:12:46.530977 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:46.511760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:59815 (local address 127.10.173.190:45765)
0811 22:12:46.512299 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:46.512305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:46.513057 (+ 752us) server_negotiation.cc:407] Connection header received
0811 22:12:46.515205 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:46.515211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:46.515324 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:46.515460 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:46.517110 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:46.518096 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:46.520076 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:46.520536 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:46.520709 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:46.524540 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:46.524545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:46.524548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:46.524585 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:46.527856 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:46.528654 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:46.528657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:46.528660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:46.528787 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:46.529277 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:46.529281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:46.529283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:46.529593 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:46.529731 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:46.530478 (+ 747us) server_negotiation.cc:299] Negotiation successful
0811 22:12:46.530662 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":278,"threads_started":1}
I20260811 22:12:46.532289 24709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:46.512115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:59815)
0811 22:12:46.512870 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:46.512909 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:46.514876 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:46.515722 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:46.515731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:46.516291 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:46.516905 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:46.516924 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:46.518274 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:46.518279 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:46.519904 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:46.519926 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:46.520846 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:46.520857 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:46.521132 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:46.521834 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:46.521860 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:46.524290 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:46.528010 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:46.528017 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:46.528032 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:46.528496 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:46.528916 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:46.528920 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:46.528923 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:46.529141 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:46.529725 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:46.529731 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:46.530047 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 22:12:46.530300 (+ 253us) client_negotiation.cc:240] Negotiation successful
0811 22:12:46.530796 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":291,"threads_started":1}
I20260811 22:12:46.533977 24706 heartbeater.cc:347] Connected to a master server at 127.10.173.190:45765
I20260811 22:12:46.534332 24706 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:46.535132 24706 heartbeater.cc:511] Master 127.10.173.190:45765 requested a full tablet report, sending...
I20260811 22:12:46.536664 23968 ts_manager.cc:194] Registered new tserver with Master: 76291fdd94fb4cf88581599dceb98e3c (127.10.173.131:35673)
I20260811 22:12:46.537565 23968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:59815
I20260811 22:12:46.545943 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab.
Entry for principal oryx/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab.
WARNING: no policy specified for oryx/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab.
Entry for principal oryx/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab.
WARNING: no policy specified for oryx/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab.
Entry for principal oryx/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab.
I20260811 22:12:46.730014 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 24072
Aug 11 22:12:46 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:12:46.748735 24720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:46.731465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38484 (local address 127.10.173.129:35143)
0811 22:12:46.732463 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:46.732471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:46.732500 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:12:46.732564 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:46.732570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:46.732806 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:46.732943 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:46.734528 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:46.735559 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:46.736872 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:46.737479 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:46.737584 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:46.742386 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:46.742401 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:46.742530 (+ 129us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:46.742574 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:46.745874 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:46.746726 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:46.746734 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:46.746737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:46.746881 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:46.747279 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:46.747283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:46.747285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:46.747669 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:46.747920 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:46.748191 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 22:12:46.748415 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":631,"threads_started":1}
I20260811 22:12:46.750062 24269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:46.838136 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 24072
I20260811 22:12:46.865135 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 24286
Aug 11 22:12:46 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:12:46.883582 24723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:46.866192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.10.173.130:38913)
0811 22:12:46.866853 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:46.866860 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:46.866888 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:12:46.866952 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:46.866959 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:46.867179 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:46.867333 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:46.868696 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:46.869822 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:46.871271 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:46.871845 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:46.871951 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:46.876874 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:46.876888 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:46.876904 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:46.876944 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:46.880300 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:46.881132 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:46.881141 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:46.881144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:46.881288 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:46.881731 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:46.881752 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:46.881754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:46.882165 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:46.882555 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:46.882958 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 22:12:46.883206 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":344,"threads_started":1}
I20260811 22:12:46.885144 24473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:46.969936 24482 heartbeater.cc:503] Master 127.10.173.190:45765 was elected leader, sending a full tablet report...
I20260811 22:12:46.971258 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 24286
W20260811 22:12:46.996091 23947 connection.cc:535] server connection from 127.10.173.130:38065 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:12:46.996470 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 24492
Aug 11 22:12:47 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:12:47.022140 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:46.997614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.10.173.131:35673)
0811 22:12:46.998232 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:46.998239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:46.998289 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:12:46.998353 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:46.998359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:46.998666 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:46.998816 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:47.000527 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:47.001677 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:47.003283 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:47.003907 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:47.004032 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:47.011940 (+ 7908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:47.011962 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:47.011979 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:47.012030 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:47.019119 (+ 7089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:47.019948 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:47.019956 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:47.019959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:47.020154 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:47.020528 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:47.020531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:47.020534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:47.020940 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:47.021225 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:47.021450 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 22:12:47.021667 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":289,"threads_started":1}
I20260811 22:12:47.023566 24696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:47.107775 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 24492
I20260811 22:12:47.135949 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 23918
I20260811 22:12:47.137352 24022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:12:47.243917 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 23918
I20260811 22:12:47.277832 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:45765
--webserver_interface=127.10.173.190
--webserver_port=36461
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:45765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:47.628518 24734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:47.628818 24734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:47.628979 24734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:47.636202 24734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:47.636308 24734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:47.636400 24734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:47.636510 24734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:47.636575 24734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:47.646334 24734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:45765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:45765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=36461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:47.647708 24734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:47.649012 24734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:47.661649 24742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:47.661566 24741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:47.661705 24745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:47.662781 24734 server_base.cc:1034] running on GCE node
I20260811 22:12:47.663609 24734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:47.665014 24734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:47.666246 24734 hybrid_clock.cc:648] HybridClock initialized: now 1786486367666190 us; error 46 us; skew 500 ppm
I20260811 22:12:47.666635 24734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:47 dist-test-slave-68xk krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486367, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:47.669888 24734 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:12:47.671566 24734 webserver.cc:460] Webserver started at http://127.10.173.190:36461/ using document root <none> and password file <none>
I20260811 22:12:47.672463 24734 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:47.672646 24734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:47.679415 24734 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 22:12:47.682694 24761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:47.684255 24734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:12:47.684449 24734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "5f9d189f72684677a30d26fbd61bcb2c"
format_stamp: "Formatted at 2026-08-11 22:12:44 on dist-test-slave-68xk"
server_key: "062d9451eb58920632d7e8d54bf4c0f0"
server_key_iv: "6d96e675e14249652bd5e47758defa09"
server_key_version: "encryptionkey@0"
I20260811 22:12:47.685467 24734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:47.711586 24734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:47.716187 24734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:47.716499 24734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:47.736505 24734 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:45765
I20260811 22:12:47.736541 24839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:45765 every 8 connection(s)
I20260811 22:12:47.738075 24734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:47.745178 24843 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:12:47.746096 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 24734
I20260811 22:12:47.747133 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:35143
--local_ip_for_outbound_sockets=127.10.173.129
--tserver_master_addrs=127.10.173.190:45765
--webserver_port=41941
--webserver_interface=127.10.173.129
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.173.129
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:47.750571 24843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c: Bootstrap starting.
I20260811 22:12:47.772542 24843 log.cc:824] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:47.782284 24843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:12:47.782912 24843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c: Bootstrap complete.
I20260811 22:12:47.793788 24843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } }
I20260811 22:12:47.794822 24843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f9d189f72684677a30d26fbd61bcb2c, State: Initialized, Role: FOLLOWER
I20260811 22:12:47.795526 24843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } }
I20260811 22:12:47.795828 24843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:47.795987 24843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:47.796223 24843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:12:47.800652 24843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } }
I20260811 22:12:47.801218 24843 leader_election.cc:302] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f9d189f72684677a30d26fbd61bcb2c; no voters:
I20260811 22:12:47.801976 24843 leader_election.cc:288] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:12:47.802138 24848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:12:47.803624 24848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [term 2 LEADER]: Becoming Leader. State: Replica: 5f9d189f72684677a30d26fbd61bcb2c, State: Running, Role: LEADER
I20260811 22:12:47.804268 24848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } }
I20260811 22:12:47.804541 24843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:47.807319 24851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f9d189f72684677a30d26fbd61bcb2c. Latest consensus state: current_term: 2 leader_uuid: "5f9d189f72684677a30d26fbd61bcb2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } } }
I20260811 22:12:47.807828 24851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:47.808527 24850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5f9d189f72684677a30d26fbd61bcb2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f9d189f72684677a30d26fbd61bcb2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45765 } } }
I20260811 22:12:47.808938 24850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:47.814916 24856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:47.823395 24856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:47.825325 24856 catalog_manager.cc:1259] Loaded cluster ID: 7446596edc524b3f90381aeb34c3db6e
I20260811 22:12:47.825526 24856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:47.830770 24856 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:47.833161 24856 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5f9d189f72684677a30d26fbd61bcb2c: Loaded TSK: 0
I20260811 22:12:47.835201 24856 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:12:48.127732 24845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:48.128062 24845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:48.128230 24845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:48.135567 24845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:48.135728 24845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:48.135907 24845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:12:48.145913 24845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.129.keytab
--principal=oryx/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:35143
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=41941
--tserver_master_addrs=127.10.173.190:45765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:48.147399 24845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:48.148751 24845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:48.163950 24884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:48.164134 24886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:48.164508 24890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:48.164844 24845 server_base.cc:1034] running on GCE node
I20260811 22:12:48.165516 24845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:48.166599 24845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:48.167838 24845 hybrid_clock.cc:648] HybridClock initialized: now 1786486368167764 us; error 68 us; skew 500 ppm
I20260811 22:12:48.168220 24845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486368, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:48.171581 24845 init.cc:377] Logged in from keytab as oryx/127.10.173.129@KRBTEST.COM (short username oryx)
I20260811 22:12:48.173153 24845 webserver.cc:460] Webserver started at http://127.10.173.129:41941/ using document root <none> and password file <none>
I20260811 22:12:48.173873 24845 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:48.174023 24845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:48.179821 24845 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:12:48.182802 24907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:48.184280 24845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:12:48.184466 24845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "170954bd1a5f43e58dea5c7b2ae558c2"
format_stamp: "Formatted at 2026-08-11 22:12:45 on dist-test-slave-68xk"
server_key: "25436719adcc34d42cafb8d746a1781b"
server_key_iv: "b34f34f02b2687e1bccecc89525e05b3"
server_key_version: "encryptionkey@0"
I20260811 22:12:48.185525 24845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:48.202654 24845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:48.207082 24845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:48.207381 24845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:48.208381 24845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:48.209740 24845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:48.209856 24845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:48.209983 24845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:48.210096 24845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:48.245090 24845 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:35143
I20260811 22:12:48.245164 25093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:35143 every 8 connection(s)
I20260811 22:12:48.246690 24845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:48.248577 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 24845
I20260811 22:12:48.249392 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:38913
--local_ip_for_outbound_sockets=127.10.173.130
--tserver_master_addrs=127.10.173.190:45765
--webserver_port=45913
--webserver_interface=127.10.173.130
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.173.130
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:12:48.278906 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:48.251121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:48081 (local address 127.10.173.190:45765)
0811 22:12:48.263028 (+ 11907us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:48.263051 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:48.263114 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:12:48.263587 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:48.263604 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:48.263988 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:48.264422 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:48.266073 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.268233 (+ 2160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.270012 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.271190 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.271439 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.277808 (+ 6369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:48081: BlockingRecv error: recv got EOF from 127.10.173.129:48081 (error 108)
Metrics: {"server-negotiator.queue_time_us":11575,"thread_start_us":11346,"threads_started":1}
I20260811 22:12:48.278962 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:48.251058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:48081)
0811 22:12:48.251909 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:48.251955 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:48.253848 (+ 1893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:48.264615 (+ 10767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:48.264624 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:48.265222 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:48.265842 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.265861 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.268423 (+ 2562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.268427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.269847 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.269861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.271429 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.271435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.271649 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.272407 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:48.272431 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:48.277472 (+ 5041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":360,"threads_started":1}
W20260811 22:12:48.279776 25095 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:45765 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.173.190:45765: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:48.289887 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:48.280485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:38087)
0811 22:12:48.280704 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:48.280727 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:48.280836 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:48.281221 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:48.281225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:48.281568 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:48.281718 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.281727 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.283361 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.283365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.284340 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.284352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.285345 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.285353 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.285679 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.286322 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:48.286341 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:48.289506 (+ 3165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 22:12:48.290010 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:48.280651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:38087 (local address 127.10.173.190:45765)
0811 22:12:48.280875 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:48.280883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:48.280907 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:12:48.280975 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:48.280983 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:48.281074 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:48.281209 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:48.281876 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.283210 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.284490 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.285131 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.285298 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.289781 (+ 4483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:38087: BlockingRecv error: recv got EOF from 127.10.173.129:38087 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:48.299763 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:48.291077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:47923)
0811 22:12:48.291284 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:48.291305 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:48.291400 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:48.291725 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:48.291729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:48.292023 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:48.292179 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.292188 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.293572 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.293576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.294654 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.294667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.295620 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.295625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.295883 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.296494 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:48.296511 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:48.299528 (+ 3017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:12:48.300100 25095 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:12:48.300194 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:48.291208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:47923 (local address 127.10.173.190:45765)
0811 22:12:48.291374 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:48.291380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:48.291399 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:12:48.291505 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:48.291510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:48.291595 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:48.291695 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:48.292332 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.293426 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.294790 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.295470 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.295594 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.299781 (+ 4187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:47923: BlockingRecv error: recv got EOF from 127.10.173.129:47923 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 22:12:48.601248 25101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:48.601526 25101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:48.601622 25101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:48.608978 25101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:48.609090 25101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:48.609210 25101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:12:48.618913 25101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.130.keytab
--principal=oryx/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:38913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=45913
--tserver_master_addrs=127.10.173.190:45765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:48.620211 25101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:48.621491 25101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:48.635078 25107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:48.635140 25108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:48.635236 25110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:48.636250 25101 server_base.cc:1034] running on GCE node
I20260811 22:12:48.636941 25101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:48.637950 25101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:48.639247 25101 hybrid_clock.cc:648] HybridClock initialized: now 1786486368639186 us; error 46 us; skew 500 ppm
I20260811 22:12:48.639581 25101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486368, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:48.642858 25101 init.cc:377] Logged in from keytab as oryx/127.10.173.130@KRBTEST.COM (short username oryx)
I20260811 22:12:48.644371 25101 webserver.cc:460] Webserver started at http://127.10.173.130:45913/ using document root <none> and password file <none>
I20260811 22:12:48.645077 25101 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:48.645255 25101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:48.650665 25101 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:48.653321 25126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:48.654672 25101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:48.654850 25101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "f3958ca56ef94d278901beb3de22c5f3"
format_stamp: "Formatted at 2026-08-11 22:12:45 on dist-test-slave-68xk"
server_key: "7233170248349aebb3c820fcf00c7be9"
server_key_iv: "058a7ed0e08e20965c83f3c4ae2c6e4f"
server_key_version: "encryptionkey@0"
I20260811 22:12:48.655800 25101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:48.690130 25101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:48.694949 25101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:48.695271 25101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:48.696358 25101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:48.698015 25101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:48.698146 25101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:48.698297 25101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:48.698396 25101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:48.733822 25101 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:38913
I20260811 22:12:48.733915 25315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:38913 every 8 connection(s)
I20260811 22:12:48.735441 25101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:48.744020 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 25101
I20260811 22:12:48.744726 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:35673
--local_ip_for_outbound_sockets=127.10.173.131
--tserver_master_addrs=127.10.173.190:45765
--webserver_port=34601
--webserver_interface=127.10.173.131
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.173.131
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:12:48.752206 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:48.739294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:33429 (local address 127.10.173.190:45765)
0811 22:12:48.739542 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:48.739550 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:48.740034 (+ 484us) server_negotiation.cc:407] Connection header received
0811 22:12:48.741916 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:48.741922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:48.742008 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:48.742156 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:48.743743 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.744582 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.746294 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.746838 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.746980 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.751932 (+ 4952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:33429: BlockingRecv error: recv got EOF from 127.10.173.130:33429 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 22:12:48.752920 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:48.739006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:33429)
0811 22:12:48.739859 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:48.739903 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:48.741635 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:48.742382 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:48.742391 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:48.743006 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:48.743566 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.743582 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.744754 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.744764 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.746138 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.746151 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.747071 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.747078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.747315 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.748018 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:48.748042 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:48.751751 (+ 3709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":349,"threads_started":1}
W20260811 22:12:48.753589 25316 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:45765 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.173.190:45765: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:48.763489 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:48.754204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:49973)
0811 22:12:48.754385 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:48.754405 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:48.754546 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:48.754883 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:48.754886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:48.755251 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:48.755391 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.755399 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.756394 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.756400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.758096 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.758123 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.759011 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.759019 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.759255 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.759889 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:48.759917 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:48.763206 (+ 3289us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 22:12:48.763824 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:48.754315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:49973 (local address 127.10.173.190:45765)
0811 22:12:48.754543 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:48.754551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:48.754574 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:12:48.754634 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:48.754641 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:48.754736 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:48.754850 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:48.755540 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.756248 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.758295 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.758859 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.758961 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.763431 (+ 4470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:49973: BlockingRecv error: recv got EOF from 127.10.173.130:49973 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:48 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:12:48.772825 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:48.764483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:48819 (local address 127.10.173.190:45765)
0811 22:12:48.764651 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:48.764657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:48.764787 (+ 130us) server_negotiation.cc:407] Connection header received
0811 22:12:48.764961 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:48.764966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:48.765040 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:48.765143 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:48.765775 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.766551 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.767883 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.768285 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.768421 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.772661 (+ 4240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:48819: BlockingRecv error: recv got EOF from 127.10.173.130:48819 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:12:48.772855 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:48.764325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:48819)
0811 22:12:48.764693 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:48.764711 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:48.764841 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:48.765169 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:48.765174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:48.765483 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:48.765628 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.765635 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.766695 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.766700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.767743 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:48.767756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:48.768431 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:48.768437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:48.768621 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:48.769208 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:48.769225 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:48.772484 (+ 3259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251}
W20260811 22:12:48.773219 25316 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:12:49.093581 25321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:49.093885 25321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:49.093983 25321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:49.101320 25321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:49.101435 25321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:49.101549 25321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:12:49.111493 25321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/krb5kdc/oryx_127.10.173.131.keytab
--principal=oryx/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:35673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=34601
--tserver_master_addrs=127.10.173.190:45765
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:12:49.112787 25321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:49.114058 25321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:49.127557 25328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:49.127486 25329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:49.127596 25332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:49.128494 25321 server_base.cc:1034] running on GCE node
I20260811 22:12:49.129113 25321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:49.130093 25321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:49.131331 25321 hybrid_clock.cc:648] HybridClock initialized: now 1786486369131293 us; error 54 us; skew 500 ppm
I20260811 22:12:49.131654 25321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486369, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:49.134850 25321 init.cc:377] Logged in from keytab as oryx/127.10.173.131@KRBTEST.COM (short username oryx)
I20260811 22:12:49.136322 25321 webserver.cc:460] Webserver started at http://127.10.173.131:34601/ using document root <none> and password file <none>
I20260811 22:12:49.137019 25321 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:49.137189 25321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:49.142737 25321 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:49.145417 25350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:49.146842 25321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:49.147027 25321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "76291fdd94fb4cf88581599dceb98e3c"
format_stamp: "Formatted at 2026-08-11 22:12:46 on dist-test-slave-68xk"
server_key: "146ae1ed8e2ff1e398a1b305eb924bca"
server_key_iv: "8c3ef1cfed1621fb4bb074c74021c610"
server_key_version: "encryptionkey@0"
I20260811 22:12:49.147971 25321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:49.168710 25321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:49.172830 25321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:49.173111 25321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:49.174192 25321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:49.175662 25321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:49.175808 25321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:49.175944 25321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:49.176039 25321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:49.212936 25321 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:35673
I20260811 22:12:49.212985 25533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:35673 every 8 connection(s)
I20260811 22:12:49.214385 25321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:49.215505 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 25321
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:12:49.233633 25538 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:49.217544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:42959 (local address 127.10.173.190:45765)
0811 22:12:49.218052 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:49.218058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:49.219418 (+ 1360us) server_negotiation.cc:407] Connection header received
0811 22:12:49.221807 (+ 2389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:49.221810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:49.221870 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:49.221962 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:49.223935 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.224736 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.227209 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.228078 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.228183 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.233345 (+ 5162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:42959: BlockingRecv error: recv got EOF from 127.10.173.131:42959 (error 108)
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":243,"threads_started":1}
I20260811 22:12:49.234193 25100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:49.216592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.10.173.190:45765)
0811 22:12:49.216858 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:49.216866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:49.217405 (+ 539us) server_negotiation.cc:407] Connection header received
0811 22:12:49.218098 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:49.218104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:49.218187 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:49.218296 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:49.220557 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.221364 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.223190 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.223616 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.223722 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.227077 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:49.227091 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:49.227106 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:49.227182 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:49.230708 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:49.231454 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:49.231463 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:49.231471 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:49.231654 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:49.232103 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:49.232108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:49.232112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:49.232543 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:49.232894 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:49.233373 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 22:12:49.233938 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:12:49.234256 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:49.218205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:42959)
0811 22:12:49.219175 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:49.219227 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:49.221320 (+ 2093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:49.222274 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:49.222285 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:49.223029 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:49.223699 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:49.223736 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.224909 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.224916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:49.227078 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:49.227089 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.228321 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.228333 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:49.228555 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.229178 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:49.229225 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:49.233108 (+ 3883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":407,"threads_started":1}
W20260811 22:12:49.234927 25534 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:45765 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.173.190:45765: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:49.243314 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:49.235571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:47947)
0811 22:12:49.235752 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:49.235773 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:49.235875 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:49.236189 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:49.236193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:49.236623 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:49.236812 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:49.236822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.238044 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.238051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:49.239158 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:49.239170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.239807 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.239811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:49.239929 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.240416 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:49.240432 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:49.243082 (+ 2650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:12:49.243501 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:49.235674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:47947 (local address 127.10.173.190:45765)
0811 22:12:49.235835 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:49.235839 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:49.235854 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:12:49.235977 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:49.235980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:49.236058 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:49.236131 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:49.236947 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.237880 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.239286 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.239684 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.239779 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.243267 (+ 3488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:47947: BlockingRecv error: recv got EOF from 127.10.173.131:47947 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:49.252168 25540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:49.243975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:45763)
0811 22:12:49.244140 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:49.244165 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:49.244283 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:49.244621 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:49.244627 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:49.244945 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:49.245116 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:49.245125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.246086 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.246091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:49.247066 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:49.247079 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.247789 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.247795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:49.247982 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.248574 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:49.248595 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:49.251841 (+ 3246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:12:49.252327 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:49.244094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:45763 (local address 127.10.173.190:45765)
0811 22:12:49.244379 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:49.244384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:49.244407 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:12:49.244443 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:49.244446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:49.244493 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:49.244593 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:49.245266 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.245946 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.247219 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.247638 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.247775 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.252024 (+ 4249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:45763: BlockingRecv error: recv got EOF from 127.10.173.131:45763 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
W20260811 22:12:49.252440 25534 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:49.310285 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:49.300778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:43029)
0811 22:12:49.301449 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:49.301466 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:49.301581 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:49.301927 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:49.301930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:49.302456 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:49.302658 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:49.302669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.303852 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.303857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:49.305000 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:49.305015 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.306027 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.306032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:49.306202 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.306944 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:49.306962 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:49.309988 (+ 3026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":428,"threads_started":1}
W20260811 22:12:49.310644 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:49.300914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:43029 (local address 127.10.173.190:45765)
0811 22:12:49.301111 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:49.301116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:49.301546 (+ 430us) server_negotiation.cc:407] Connection header received
0811 22:12:49.301703 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:49.301707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:49.301774 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:49.301875 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:49.302840 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.303682 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.305216 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.305810 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.305965 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.310329 (+ 4364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:43029: BlockingRecv error: recv got EOF from 127.10.173.129:43029 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:49 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:49.782969 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:49.773942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:53233)
0811 22:12:49.774627 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:49.774643 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:49.774811 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:49.775229 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:49.775234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:49.775775 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:49.775985 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:49.775995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.777061 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.777069 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:49.778053 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:49.778067 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.778928 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.778935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:49.779110 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.779750 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:49.779768 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:49.782748 (+ 2980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":410,"threads_started":1}
W20260811 22:12:49.783273 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:49.774037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:53233 (local address 127.10.173.190:45765)
0811 22:12:49.774244 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:49.774253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:49.774741 (+ 488us) server_negotiation.cc:407] Connection header received
0811 22:12:49.774969 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:49.774973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:49.775060 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:49.775188 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:49.776145 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.776906 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.778205 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:49.778728 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:49.778865 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:49.782980 (+ 4115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:53233: BlockingRecv error: recv got EOF from 127.10.173.130:53233 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 22:12:50 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:50 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:50.262239 25546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:50.253006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:52653)
0811 22:12:50.253539 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:50.253556 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:50.253677 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:50.254008 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:50.254011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:50.254500 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:50.254706 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:50.254715 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.255843 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.255847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:50.256803 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:50.256814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.257542 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.257546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:50.257668 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:50.258270 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:50.258287 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:50.261903 (+ 3616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":283,"threads_started":1}
W20260811 22:12:50.262362 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:50.253083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:52653 (local address 127.10.173.190:45765)
0811 22:12:50.253336 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:50.253340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:50.253645 (+ 305us) server_negotiation.cc:407] Connection header received
0811 22:12:50.253790 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:50.253794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:50.253857 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:50.253944 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:50.254881 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.255694 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.256933 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.257385 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.257482 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:50.262112 (+ 4630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:52653: BlockingRecv error: recv got EOF from 127.10.173.131:52653 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 22:12:50 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:50 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:50.320770 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:50.311630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:56383)
0811 22:12:50.312207 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:50.312224 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:50.312345 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:50.312719 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:50.312724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:50.313185 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:50.313377 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:50.313387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.314443 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.314448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:50.315484 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:50.315509 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.316220 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.316225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:50.316352 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:50.317125 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:50.317150 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:50.320395 (+ 3245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":318,"threads_started":1}
W20260811 22:12:50.321038 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:50.311665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:56383 (local address 127.10.173.190:45765)
0811 22:12:50.311875 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:50.311880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:50.312301 (+ 421us) server_negotiation.cc:407] Connection header received
0811 22:12:50.312441 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:50.312445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:50.312522 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:50.312606 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:50.313513 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.314260 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.315643 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.316058 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.316152 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:50.320643 (+ 4491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:56383: BlockingRecv error: recv got EOF from 127.10.173.129:56383 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 22:12:50 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:50 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:50.792953 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:50.784038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:47685)
0811 22:12:50.784726 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:50.784744 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:50.784871 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:50.785274 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:50.785278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:50.785730 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:50.785916 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:50.785926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.787036 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.787041 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:50.787999 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:50.788012 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.788720 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.788725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:50.788844 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:50.789624 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:50.789642 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:50.792669 (+ 3027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":403,"threads_started":1}
W20260811 22:12:50.793228 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:50.784086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:47685 (local address 127.10.173.190:45765)
0811 22:12:50.784377 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:50.784381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:50.784831 (+ 450us) server_negotiation.cc:407] Connection header received
0811 22:12:50.784984 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:50.784988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:50.785083 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:50.785167 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:50.786057 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.786882 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.788148 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:50.788548 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:50.788659 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:50.792962 (+ 4303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:47685: BlockingRecv error: recv got EOF from 127.10.173.130:47685 (error 108)
Metrics: {"server-negotiator.queue_time_us":173}
Aug 11 22:12:51 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:51 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:51.271876 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:51.263249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:48365)
0811 22:12:51.263867 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:51.263890 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:51.264036 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:51.264416 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:51.264421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:51.264861 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:51.265052 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:51.265063 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.266103 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.266109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:51.267103 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:51.267116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.267875 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.267881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:51.268075 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:51.268641 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:51.268659 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:51.271620 (+ 2961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":333,"threads_started":1}
W20260811 22:12:51.272064 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:51.263450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:48365 (local address 127.10.173.190:45765)
0811 22:12:51.263651 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:51.263657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:51.263967 (+ 310us) server_negotiation.cc:407] Connection header received
0811 22:12:51.264175 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:51.264181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:51.264257 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:51.264372 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:51.265214 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.265957 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.267253 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.267704 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.267858 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:51.271815 (+ 3957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:48365: BlockingRecv error: recv got EOF from 127.10.173.131:48365 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 22:12:51 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:51 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:51.330315 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:51.321763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:58031)
0811 22:12:51.322405 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:51.322455 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:51.322637 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:51.323010 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:51.323017 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:51.323421 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:51.323595 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:51.323604 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.324679 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.324685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:51.325604 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:51.325615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.326362 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.326367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:51.326576 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:51.327178 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:51.327196 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:51.330040 (+ 2844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":370,"threads_started":1}
W20260811 22:12:51.330506 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:51.321947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:58031 (local address 127.10.173.190:45765)
0811 22:12:51.322236 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:51.322242 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:51.322572 (+ 330us) server_negotiation.cc:407] Connection header received
0811 22:12:51.322777 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:51.322783 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:51.322858 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:51.322968 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:51.323759 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.324515 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.325759 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.326198 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.326349 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:51.330241 (+ 3892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:58031: BlockingRecv error: recv got EOF from 127.10.173.129:58031 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
Aug 11 22:12:51 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:51 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:51.802621 25554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:51.794065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:36865)
0811 22:12:51.794646 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:51.794665 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:51.794795 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:51.795177 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:51.795181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:51.795585 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:51.795772 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:51.795781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.796804 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.796809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:51.797781 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:51.797796 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.798515 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.798529 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:51.798689 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:51.799298 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:51.799316 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:51.802288 (+ 2972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":287,"threads_started":1}
W20260811 22:12:51.802784 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:51.794133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:36865 (local address 127.10.173.190:45765)
0811 22:12:51.794401 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:51.794436 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:51.794774 (+ 338us) server_negotiation.cc:407] Connection header received
0811 22:12:51.794917 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:51.794920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:51.795019 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:51.795112 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:51.795909 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.796650 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.797921 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:51.798335 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:51.798511 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:51.802556 (+ 4045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:36865: BlockingRecv error: recv got EOF from 127.10.173.130:36865 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 22:12:52 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:52 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:52.284870 25556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:52.272947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:57661)
0811 22:12:52.273552 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:52.273572 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:52.273755 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:52.274135 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:52.274140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:52.274586 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:52.274772 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:52.274782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.275763 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.275771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:52.276678 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:52.276689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.277417 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.277422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:52.277599 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:52.281501 (+ 3902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:52.281521 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:52.284486 (+ 2965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":331,"threads_started":1}
W20260811 22:12:52.285000 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:52.273036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:57661 (local address 127.10.173.190:45765)
0811 22:12:52.273283 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:52.273293 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:52.273659 (+ 366us) server_negotiation.cc:407] Connection header received
0811 22:12:52.273884 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:52.273889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:52.273989 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:52.274096 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:52.274917 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.275627 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.276823 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.277272 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.277405 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:52.284750 (+ 7345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:57661: BlockingRecv error: recv got EOF from 127.10.173.131:57661 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 22:12:52 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:52 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:52.340374 25557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:52.331316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:45967)
0811 22:12:52.331806 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:52.331824 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:52.331945 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:52.332275 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:52.332279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:52.332674 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:52.332832 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:52.332841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.333884 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.333888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:52.335381 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:52.335405 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.336184 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.336191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:52.336382 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:52.336984 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:52.337002 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:52.340113 (+ 3111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":283,"threads_started":1}
W20260811 22:12:52.340747 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:52.331383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:45967 (local address 127.10.173.190:45765)
0811 22:12:52.331632 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:52.331639 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:52.331904 (+ 265us) server_negotiation.cc:407] Connection header received
0811 22:12:52.332045 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:52.332049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:52.332111 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:52.332193 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:52.332961 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.333706 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.335555 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.336007 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.336152 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:52.340331 (+ 4179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:45967: BlockingRecv error: recv got EOF from 127.10.173.129:45967 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 22:12:52 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:52 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:52.813005 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:52.803741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:54245)
0811 22:12:52.804274 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:52.804292 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:52.804407 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:52.804804 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:52.804810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:52.805326 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:52.805501 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:52.805512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.806808 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.806817 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:52.807892 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:52.807904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.808839 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.808844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:52.809051 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:52.809708 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:52.809727 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:52.812664 (+ 2937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":299,"threads_started":1}
W20260811 22:12:52.813164 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:52.803793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:54245 (local address 127.10.173.190:45765)
0811 22:12:52.804002 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:52.804007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:52.804369 (+ 362us) server_negotiation.cc:407] Connection header received
0811 22:12:52.804515 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:52.804519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:52.804603 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:52.804761 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:52.805683 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.806658 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.808052 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:52.808671 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:52.808830 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:52.812908 (+ 4078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:54245: BlockingRecv error: recv got EOF from 127.10.173.130:54245 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 22:12:53 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:53 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:53.295342 25559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:53.285937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:37855)
0811 22:12:53.286490 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:53.286507 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:53.286627 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:53.286962 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:53.286965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:53.287362 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:53.287522 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:53.287531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.288564 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.288567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:53.289646 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:53.289659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.290646 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.290654 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:53.290814 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:53.291455 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:53.291473 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:53.294973 (+ 3500us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":314,"threads_started":1}
W20260811 22:12:53.295522 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:53.285979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:37855 (local address 127.10.173.190:45765)
0811 22:12:53.286172 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:53.286178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:53.286613 (+ 435us) server_negotiation.cc:407] Connection header received
0811 22:12:53.286729 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:53.286733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:53.286816 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:53.286908 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:53.287690 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.288434 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.289775 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.290218 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.290622 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:53.295182 (+ 4560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:37855: BlockingRecv error: recv got EOF from 127.10.173.131:37855 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 22:12:53 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:53 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:53.349442 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:53.341460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:50275)
0811 22:12:53.342005 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:53.342021 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:53.342127 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:53.342388 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:53.342391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:53.342878 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:53.343047 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:53.343056 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.344051 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.344056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:53.344980 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:53.344992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.345682 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.345694 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:53.345831 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:53.346391 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:53.346429 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:53.349167 (+ 2738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":267,"threads_started":1}
W20260811 22:12:53.349579 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:53.341536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:50275 (local address 127.10.173.190:45765)
0811 22:12:53.341799 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:53.341805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:53.342101 (+ 296us) server_negotiation.cc:407] Connection header received
0811 22:12:53.342227 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:53.342231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:53.342293 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:53.342374 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:53.343187 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.343918 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.345129 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.345551 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.345648 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:53.349374 (+ 3726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:50275: BlockingRecv error: recv got EOF from 127.10.173.129:50275 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 22:12:53 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:53 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:53.822227 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:53.814167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:34723)
0811 22:12:53.814803 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:53.814819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:53.814956 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:53.815383 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:53.815388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:53.815883 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:53.816048 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:53.816057 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.817086 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.817089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:53.817997 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:53.818007 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.818703 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.818707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:53.818824 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:53.819351 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:53.819368 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:53.821996 (+ 2628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":417,"threads_started":1}
W20260811 22:12:53.822490 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:53.814361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:34723 (local address 127.10.173.190:45765)
0811 22:12:53.814576 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:53.814583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:53.814911 (+ 328us) server_negotiation.cc:407] Connection header received
0811 22:12:53.815113 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:53.815117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:53.815220 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:53.815329 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:53.816199 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.816969 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.818133 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:53.818583 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:53.818678 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:53.822193 (+ 3515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:34723: BlockingRecv error: recv got EOF from 127.10.173.130:34723 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 22:12:54 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:54 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:54.305114 25565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:54.296306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:55909)
0811 22:12:54.296868 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:54.296886 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:54.296998 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:54.297340 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:54.297345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:54.297863 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:54.298035 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:54.298045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.299086 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.299090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:54.300195 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:54.300214 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.301117 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.301126 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:54.301264 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:54.301843 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:54.301861 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:54.304722 (+ 2861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":286,"threads_started":1}
W20260811 22:12:54.305214 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:54.296392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:55909 (local address 127.10.173.190:45765)
0811 22:12:54.296562 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:54.296567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:54.296959 (+ 392us) server_negotiation.cc:407] Connection header received
0811 22:12:54.297094 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:54.297098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:54.297167 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:54.297302 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:54.298200 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.298951 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.300350 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.300940 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.301079 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:54.304920 (+ 3841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:55909: BlockingRecv error: recv got EOF from 127.10.173.131:55909 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 22:12:54 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:54 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:54.359086 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:54.350543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:47519)
0811 22:12:54.351116 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:54.351133 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:54.351299 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:54.351699 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:54.351702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:54.352095 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:54.352297 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:54.352306 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.353448 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.353452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:54.354357 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:54.354369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.355191 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.355195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:54.355316 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:54.355928 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:54.355946 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:54.358868 (+ 2922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":329,"threads_started":1}
W20260811 22:12:54.359436 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:54.350611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:47519 (local address 127.10.173.190:45765)
0811 22:12:54.350810 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:54.350814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:54.351218 (+ 404us) server_negotiation.cc:407] Connection header received
0811 22:12:54.351420 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:54.351424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:54.351516 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:54.351632 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:54.352488 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.353296 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.354559 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.355037 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.355179 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:54.359178 (+ 3999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:47519: BlockingRecv error: recv got EOF from 127.10.173.129:47519 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 22:12:54 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:54 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:54.832926 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:54.823500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:34975)
0811 22:12:54.824281 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:54.824298 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:54.824444 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:54.824985 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:54.824990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:54.825582 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:54.825754 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:54.825763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.826958 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.826963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:54.827880 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:54.827892 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.828930 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.828934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:54.829070 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:54.829772 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:54.829791 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:54.832705 (+ 2914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":584,"threads_started":1}
W20260811 22:12:54.833248 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:54.823657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:34975 (local address 127.10.173.190:45765)
0811 22:12:54.823924 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:54.823933 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:54.824401 (+ 468us) server_negotiation.cc:407] Connection header received
0811 22:12:54.824665 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:54.824671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:54.824800 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:54.824921 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:54.825899 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.826798 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.828015 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:54.828740 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:54.828861 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:54.832936 (+ 4075us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:34975: BlockingRecv error: recv got EOF from 127.10.173.130:34975 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 11 22:12:55 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:55 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:55.316449 25568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:55.306185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:52945)
0811 22:12:55.306997 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:55.307016 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:55.307193 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:55.307574 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:55.307579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:55.308095 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:55.308263 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:55.308273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.309445 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.309458 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:55.310565 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:55.310579 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.311429 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.311436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:55.311679 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:55.312514 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:55.312543 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:55.316085 (+ 3542us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":532,"threads_started":1}
W20260811 22:12:55.316583 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:55.306341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:52945 (local address 127.10.173.190:45765)
0811 22:12:55.306610 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:55.306620 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:55.307120 (+ 500us) server_negotiation.cc:407] Connection header received
0811 22:12:55.307336 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:55.307349 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:55.307416 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:55.307532 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:55.308431 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.309274 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.310732 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.311246 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.311402 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:55.316325 (+ 4923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:52945: BlockingRecv error: recv got EOF from 127.10.173.131:52945 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 22:12:55 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:55 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:55.369864 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:55.360238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:36623)
0811 22:12:55.360991 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:55.361010 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:55.361201 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:55.361604 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:55.361609 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:55.362127 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:55.362321 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:55.362331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.363532 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.363539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:55.364597 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:55.364610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.365465 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.365472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:55.365668 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:55.366369 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:55.366389 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:55.369499 (+ 3110us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":371,"threads_started":1}
W20260811 22:12:55.369989 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:55.360301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:36623 (local address 127.10.173.190:45765)
0811 22:12:55.360499 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:55.360504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:55.361119 (+ 615us) server_negotiation.cc:407] Connection header received
0811 22:12:55.361340 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:55.361347 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:55.361434 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:55.361564 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:55.362516 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.363362 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.364775 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.365266 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.365424 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:55.369746 (+ 4322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:36623: BlockingRecv error: recv got EOF from 127.10.173.129:36623 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 22:12:55 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:55 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:55.843132 25570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:55.834116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:53859)
0811 22:12:55.834739 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:55.834763 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:55.834877 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:55.835269 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:55.835273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:55.835770 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:55.835949 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:55.835957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.837052 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.837056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:55.837985 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:55.837998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.838773 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.838778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:55.838922 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:55.839553 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:55.839575 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:55.842861 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":338,"threads_started":1}
W20260811 22:12:55.843397 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:55.834195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:53859 (local address 127.10.173.190:45765)
0811 22:12:55.834494 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:55.834504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:55.834851 (+ 347us) server_negotiation.cc:407] Connection header received
0811 22:12:55.835030 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:55.835034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:55.835123 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:55.835215 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:55.836089 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.836915 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.838140 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:55.838596 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:55.838707 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:55.843094 (+ 4387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:53859: BlockingRecv error: recv got EOF from 127.10.173.130:53859 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 11 22:12:56 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:56 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:56.327014 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:56.317571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:39745)
0811 22:12:56.318254 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:56.318274 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:56.318435 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:56.318808 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:56.318812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:56.319309 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:56.319497 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:56.319506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.320522 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.320526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:56.321755 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:56.321775 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.322818 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.322824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:56.323012 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:56.323749 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:56.323776 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:56.326755 (+ 2979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":285,"threads_started":1}
W20260811 22:12:56.327275 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:56.317622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:39745 (local address 127.10.173.190:45765)
0811 22:12:56.317848 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:56.317857 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:56.318365 (+ 508us) server_negotiation.cc:407] Connection header received
0811 22:12:56.318557 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:56.318561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:56.318634 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:56.318741 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:56.319643 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.320376 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.321932 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.322606 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.322782 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:56.326998 (+ 4216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:39745: BlockingRecv error: recv got EOF from 127.10.173.131:39745 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 22:12:56 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:56 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:56.380208 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:56.371090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:32853)
0811 22:12:56.371725 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:56.371747 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:56.371924 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:56.372355 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:56.372363 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:56.372925 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:56.373181 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:56.373197 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.374270 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.374276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:56.375267 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:56.375279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.376051 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.376057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:56.376245 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:56.376905 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:56.376923 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:56.379864 (+ 2941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":307,"threads_started":1}
W20260811 22:12:56.380329 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:56.371320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:32853 (local address 127.10.173.190:45765)
0811 22:12:56.371550 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:56.371556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:56.371851 (+ 295us) server_negotiation.cc:407] Connection header received
0811 22:12:56.372082 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:56.372087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:56.372181 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:56.372305 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:56.373337 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.374101 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.375414 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.375867 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.376019 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:56.380077 (+ 4058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:32853: BlockingRecv error: recv got EOF from 127.10.173.129:32853 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 11 22:12:56 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:56 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:56.853963 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:56.844195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:54011)
0811 22:12:56.844865 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:56.844883 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:56.845075 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:56.845522 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:56.845526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:56.846193 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:56.846457 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:56.846472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.847861 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.847865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:56.848853 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:56.848867 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.849695 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.849700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:56.849826 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:56.850545 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:56.850564 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:56.853727 (+ 3163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":351,"threads_started":1}
W20260811 22:12:56.854228 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:56.844373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:54011 (local address 127.10.173.190:45765)
0811 22:12:56.844621 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:56.844630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:56.844982 (+ 352us) server_negotiation.cc:407] Connection header received
0811 22:12:56.845266 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:56.845270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:56.845337 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:56.845481 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:56.846672 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.847713 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.849010 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:56.849479 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:56.849583 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:56.853980 (+ 4397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:54011: BlockingRecv error: recv got EOF from 127.10.173.130:54011 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 22:12:57 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:57 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:57.340197 25575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:57.328056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:38019)
0811 22:12:57.328668 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:57.328686 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:57.328812 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:57.329123 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:57.329126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:57.329561 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:57.329723 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:57.329731 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.330839 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.330844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:57.331796 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:57.331809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.332601 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.332607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:57.332785 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:57.336470 (+ 3685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:57.336492 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:57.339933 (+ 3441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":328,"threads_started":1}
W20260811 22:12:57.340575 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:57.328133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:38019 (local address 127.10.173.190:45765)
0811 22:12:57.328342 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:57.328346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:57.328774 (+ 428us) server_negotiation.cc:407] Connection header received
0811 22:12:57.328917 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:57.328921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:57.328987 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:57.329073 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:57.329858 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.330670 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.331946 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.332410 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.332550 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:57.340269 (+ 7719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:38019: BlockingRecv error: recv got EOF from 127.10.173.131:38019 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 22:12:57 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:57 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:57.390141 25576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:57.381173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:58721)
0811 22:12:57.381746 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:57.381767 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:57.381885 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:57.382181 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:57.382185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:57.382671 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:57.382864 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:57.382873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.383865 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.383869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:57.384814 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:57.384826 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.385549 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.385554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:57.385714 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:57.386337 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:57.386358 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:57.389771 (+ 3413us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":327,"threads_started":1}
W20260811 22:12:57.390271 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:57.381278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:58721 (local address 127.10.173.190:45765)
0811 22:12:57.381526 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:57.381533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:57.381848 (+ 315us) server_negotiation.cc:407] Connection header received
0811 22:12:57.381983 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:57.381987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:57.382049 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:57.382129 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:57.382995 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.383756 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.384938 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.385377 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.385472 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:57.390021 (+ 4549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:58721: BlockingRecv error: recv got EOF from 127.10.173.129:58721 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 22:12:57 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:57 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:57.864125 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:57.855070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:41633)
0811 22:12:57.855662 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:57.855679 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:57.855851 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:57.856237 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:57.856240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:57.856639 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:57.856823 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:57.856831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.857870 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.857876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:57.859143 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:57.859159 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.860000 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.860005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:57.860191 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:57.860879 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:57.860898 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:57.863856 (+ 2958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":325,"threads_started":1}
W20260811 22:12:57.864215 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:57.855134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:41633 (local address 127.10.173.190:45765)
0811 22:12:57.855349 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:57.855354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:57.855824 (+ 470us) server_negotiation.cc:407] Connection header received
0811 22:12:57.855998 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:57.856002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:57.856065 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:57.856149 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:57.856961 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.857716 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.859291 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:57.859853 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:57.859950 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:57.864033 (+ 4083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:41633: BlockingRecv error: recv got EOF from 127.10.173.130:41633 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 22:12:58 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:58 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:58.350661 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:58.341231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:53175)
0811 22:12:58.341837 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:58.341854 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:58.341972 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:58.342327 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:58.342331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:58.342812 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:58.343007 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:58.343022 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.344145 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.344149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:58.345142 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:58.345155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.345858 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.345863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:58.345994 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:58.346778 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:58.346797 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:58.350311 (+ 3514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":347,"threads_started":1}
W20260811 22:12:58.350864 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:58.341331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:53175 (local address 127.10.173.190:45765)
0811 22:12:58.341519 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:58.341523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:58.341931 (+ 408us) server_negotiation.cc:407] Connection header received
0811 22:12:58.342071 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:58.342075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:58.342152 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:58.342236 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:58.343194 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.343968 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.345297 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.345706 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.345804 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:58.350624 (+ 4820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:53175: BlockingRecv error: recv got EOF from 127.10.173.131:53175 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 22:12:58 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:58 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:58.400686 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:58.391367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:52983)
0811 22:12:58.391913 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:58.391929 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:58.392044 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:58.392433 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:58.392437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:58.392857 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:58.393024 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:58.393035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.394198 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.394203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:58.395378 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:58.395393 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.396311 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.396319 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:58.396476 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:58.397272 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:58.397290 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:58.400423 (+ 3133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":330,"threads_started":1}
W20260811 22:12:58.400918 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:58.391485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:52983 (local address 127.10.173.190:45765)
0811 22:12:58.391690 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:58.391695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:58.392040 (+ 345us) server_negotiation.cc:407] Connection header received
0811 22:12:58.392158 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:58.392162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:58.392234 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:58.392328 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:58.393234 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.394017 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.395576 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.396132 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.396248 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:58.400627 (+ 4379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:52983: BlockingRecv error: recv got EOF from 127.10.173.129:52983 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 22:12:58 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:58 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:58.873680 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:58.865231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:46531)
0811 22:12:58.865790 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:58.865807 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:58.865921 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:58.866245 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:58.866249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:58.866709 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:58.866865 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:58.866874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.868033 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.868036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:58.869011 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:58.869023 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.869731 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.869735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:58.869857 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:58.870512 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:58.870530 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:58.873435 (+ 2905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":317,"threads_started":1}
W20260811 22:12:58.873914 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:58.865317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:46531 (local address 127.10.173.190:45765)
0811 22:12:58.865520 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:58.865525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:58.865891 (+ 366us) server_negotiation.cc:407] Connection header received
0811 22:12:58.866030 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:58.866034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:58.866098 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:58.866185 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:58.867066 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.867913 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.869142 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:58.869592 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:58.869689 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:58.873646 (+ 3957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:46531: BlockingRecv error: recv got EOF from 127.10.173.130:46531 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 22:12:59 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:59 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:59.360443 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:59.351880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:49859)
0811 22:12:59.352371 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:59.352388 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:59.352503 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:59.352809 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:59.352813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:59.353237 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:59.353400 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:59.353408 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.354441 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.354445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:59.355384 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:59.355395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.356105 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.356109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:59.356236 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:59.356826 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:59.356844 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:59.360101 (+ 3257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":273,"threads_started":1}
W20260811 22:12:59.360664 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:59.351977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:49859 (local address 127.10.173.190:45765)
0811 22:12:59.352304 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:59.352313 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:59.352512 (+ 199us) server_negotiation.cc:407] Connection header received
0811 22:12:59.352608 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:59.352612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:59.352679 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:59.352766 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:59.353564 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.354286 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.355540 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.355964 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.356059 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:59.360388 (+ 4329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:49859: BlockingRecv error: recv got EOF from 127.10.173.131:49859 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
Aug 11 22:12:59 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:59 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:59.411028 25586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:59.401749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:45061)
0811 22:12:59.402356 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:59.402372 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:59.402571 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:59.402978 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:59.402981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:59.403390 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:59.403579 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:59.403587 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.404605 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.404609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:59.405621 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:59.405634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.406568 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.406574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:59.406752 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:59.407495 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:59.407513 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:59.410786 (+ 3273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":367,"threads_started":1}
W20260811 22:12:59.411190 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:59.401816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:45061 (local address 127.10.173.190:45765)
0811 22:12:59.402209 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:59.402216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:59.402471 (+ 255us) server_negotiation.cc:407] Connection header received
0811 22:12:59.402753 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:59.402758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:59.402842 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:59.402937 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:59.403713 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.404467 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.405788 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.406346 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.406531 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:59.410972 (+ 4441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:45061: BlockingRecv error: recv got EOF from 127.10.173.129:45061 (error 108)
Metrics: {"server-negotiator.queue_time_us":189}
Aug 11 22:12:59 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:12:59 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:12:59.884184 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:59.874615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:60265)
0811 22:12:59.875205 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:59.875222 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:59.875346 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:59.875694 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:59.875698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:59.876126 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:59.876347 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:59.876357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.877455 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.877460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:59.878557 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:59.878572 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.879742 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.879749 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:59.879904 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:59.880673 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:59.880707 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:59.883815 (+ 3108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":316,"threads_started":1}
W20260811 22:12:59.884284 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:59.874732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:60265 (local address 127.10.173.190:45765)
0811 22:12:59.874927 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:59.874932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:59.875300 (+ 368us) server_negotiation.cc:407] Connection header received
0811 22:12:59.875459 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:59.875463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:59.875527 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:59.875631 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:59.876487 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.877310 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.878719 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:59.879406 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:59.879571 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:59.884024 (+ 4453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:60265: BlockingRecv error: recv got EOF from 127.10.173.130:60265 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 22:13:00 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:00 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:00.370218 25589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:00.361458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:40537)
0811 22:13:00.361960 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:00.361977 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:00.362100 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:00.362522 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:00.362525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:00.362955 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:00.363138 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:00.363147 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.364115 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.364118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:00.365060 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:00.365072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.365683 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.365688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:00.365813 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:00.366694 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:00.366721 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:00.369979 (+ 3258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":271,"threads_started":1}
W20260811 22:13:00.370411 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:00.361521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:40537 (local address 127.10.173.190:45765)
0811 22:13:00.361788 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:00.361795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:00.362071 (+ 276us) server_negotiation.cc:407] Connection header received
0811 22:13:00.362247 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:00.362251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:00.362325 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:00.362449 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:00.363288 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.364000 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.365179 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.365543 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.365631 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:00.370168 (+ 4537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:40537: BlockingRecv error: recv got EOF from 127.10.173.131:40537 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 22:13:00 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:00 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:00.420596 25592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:00.411955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:33327)
0811 22:13:00.412451 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:00.412469 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:00.412604 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:00.412918 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:00.412921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:00.413329 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:00.413483 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:00.413490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.414538 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.414542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:00.415758 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:00.415777 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.416575 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.416580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:00.416728 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:00.417323 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:00.417340 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:00.420335 (+ 2995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":294,"threads_started":1}
W20260811 22:13:00.420770 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:00.412067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:33327 (local address 127.10.173.190:45765)
0811 22:13:00.412331 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:00.412338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:00.412553 (+ 215us) server_negotiation.cc:407] Connection header received
0811 22:13:00.412706 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:00.412710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:00.412783 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:00.412863 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:00.413607 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.414369 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.415927 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.416422 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.416517 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:00.420545 (+ 4028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:33327: BlockingRecv error: recv got EOF from 127.10.173.129:33327 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 11 22:13:00 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:00 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:00.894518 25594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:00.885296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:38217)
0811 22:13:00.885855 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:00.885872 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:00.886042 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:00.886476 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:00.886481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:00.886930 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:00.887093 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:00.887101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.888141 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.888147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:00.889423 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:00.889438 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.890556 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.890560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:00.890740 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:00.891418 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:00.891437 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:00.894248 (+ 2811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":301,"threads_started":1}
W20260811 22:13:00.894878 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:00.885370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:38217 (local address 127.10.173.190:45765)
0811 22:13:00.885649 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:00.885656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:00.885970 (+ 314us) server_negotiation.cc:407] Connection header received
0811 22:13:00.886226 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:00.886230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:00.886295 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:00.886406 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:00.887249 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.887989 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.889592 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:00.890358 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:00.890571 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:00.894598 (+ 4027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:38217: BlockingRecv error: recv got EOF from 127.10.173.130:38217 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 11 22:13:01 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:01 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:01.380589 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:01.371336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:40251)
0811 22:13:01.371869 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:01.371887 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:01.372011 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:01.372526 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:01.372530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:01.373015 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:01.373309 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:01.373332 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.374754 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.374760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:01.375716 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:01.375745 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.376488 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.376492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:01.376628 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:01.377247 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:01.377265 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:01.380319 (+ 3054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":324,"threads_started":1}
W20260811 22:13:01.380896 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:01.371377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:40251 (local address 127.10.173.190:45765)
0811 22:13:01.371664 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:01.371674 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:01.371982 (+ 308us) server_negotiation.cc:407] Connection header received
0811 22:13:01.372292 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:01.372296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:01.372365 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:01.372449 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:01.373487 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.374541 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.375894 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.376355 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.376462 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:01.380538 (+ 4076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:40251: BlockingRecv error: recv got EOF from 127.10.173.131:40251 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 22:13:01 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:01 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:01.430953 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:01.421793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:50949)
0811 22:13:01.422610 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:01.422632 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:01.422806 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:01.423180 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:01.423184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:01.423702 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:01.423865 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:01.423874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.424923 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.424929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:01.425878 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:01.425891 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.426768 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.426775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:01.426940 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:01.427602 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:01.427621 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:01.430581 (+ 2960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":483,"threads_started":1}
W20260811 22:13:01.431044 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:01.421877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:50949 (local address 127.10.173.190:45765)
0811 22:13:01.422214 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:01.422224 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:01.422748 (+ 524us) server_negotiation.cc:407] Connection header received
0811 22:13:01.422950 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:01.422963 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:01.423032 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:01.423143 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:01.424018 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.424782 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.426038 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.426567 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.426701 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:01.430792 (+ 4091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:50949: BlockingRecv error: recv got EOF from 127.10.173.129:50949 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 22:13:01 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:01 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:01.904425 25598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:01.895650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:46251)
0811 22:13:01.896475 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:01.896497 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:01.896661 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:01.897013 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:01.897017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:01.897434 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:01.897596 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:01.897604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.898835 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.898839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:01.899754 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:01.899765 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.900468 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.900473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:01.900583 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:01.901238 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:01.901256 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:01.904099 (+ 2843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":473,"threads_started":1}
W20260811 22:13:01.904610 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:01.895771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:46251 (local address 127.10.173.190:45765)
0811 22:13:01.896038 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:01.896059 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:01.896601 (+ 542us) server_negotiation.cc:407] Connection header received
0811 22:13:01.896795 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:01.896799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:01.896881 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:01.896972 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:01.897734 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.898692 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.899902 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:01.900309 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:01.900415 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:01.904342 (+ 3927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:46251: BlockingRecv error: recv got EOF from 127.10.173.130:46251 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 22:13:02 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:02 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:02.394551 25600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:02.381640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:35751)
0811 22:13:02.382205 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:02.382222 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:02.382390 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:02.382772 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:02.382779 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:02.383213 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:02.383378 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:02.383387 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.384459 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.384465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:02.385420 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:02.385432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.386219 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.386225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:02.386396 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:02.391101 (+ 4705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:02.391132 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:02.394224 (+ 3092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":285,"threads_started":1}
W20260811 22:13:02.394882 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:02.381718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:35751 (local address 127.10.173.190:45765)
0811 22:13:02.381981 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:02.381989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:02.382317 (+ 328us) server_negotiation.cc:407] Connection header received
0811 22:13:02.382539 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:02.382545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:02.382622 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:02.382729 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:02.383544 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.384312 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.385575 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.386028 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.386177 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:02.394561 (+ 8384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:35751: BlockingRecv error: recv got EOF from 127.10.173.131:35751 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 22:13:02 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:02 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:02.441329 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:02.432122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:45763)
0811 22:13:02.432719 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:02.432755 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:02.432886 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:02.433226 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:02.433230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:02.433676 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:02.433878 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:02.433887 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.435048 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.435085 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:02.436029 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:02.436042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.436721 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.436725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:02.436854 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:02.437629 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:02.437656 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:02.441077 (+ 3421us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":349,"threads_started":1}
W20260811 22:13:02.441556 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:02.432281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:45763 (local address 127.10.173.190:45765)
0811 22:13:02.432538 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:02.432543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:02.432840 (+ 297us) server_negotiation.cc:407] Connection header received
0811 22:13:02.432993 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:02.432997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:02.433059 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:02.433145 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:02.434030 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.434882 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.436174 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.436584 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.436679 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:02.441299 (+ 4620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:45763: BlockingRecv error: recv got EOF from 127.10.173.129:45763 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
Aug 11 22:13:02 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:02 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:02.915124 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:02.905599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:60027)
0811 22:13:02.906225 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:02.906242 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:02.906384 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:02.906787 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:02.906790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:02.907220 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:02.907404 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:02.907412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.908944 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.908948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:02.909967 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:02.909981 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.910867 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.910871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:02.910992 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:02.911687 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:02.911705 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:02.914775 (+ 3070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":311,"threads_started":1}
W20260811 22:13:02.915277 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:02.905732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:60027 (local address 127.10.173.190:45765)
0811 22:13:02.906051 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:02.906058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:02.906327 (+ 269us) server_negotiation.cc:407] Connection header received
0811 22:13:02.906558 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:02.906570 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:02.906639 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:02.906730 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:02.907614 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.908775 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.910122 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.910721 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.910836 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:02.915021 (+ 4185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:60027: BlockingRecv error: recv got EOF from 127.10.173.130:60027 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 11 22:13:03 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:03 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.131@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:03.404633 25608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.395491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.131:35163)
0811 22:13:03.396012 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:03.396028 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:03.396152 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:03.396447 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:03.396450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:03.396829 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:03.397006 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.397015 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.398535 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.398540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.399716 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.399731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.400481 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.400487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.400624 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.401266 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:03.401284 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:03.404346 (+ 3062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":309,"threads_started":1}
W20260811 22:13:03.404917 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:03.395610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:35163 (local address 127.10.173.190:45765)
0811 22:13:03.395906 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.395912 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.396118 (+ 206us) server_negotiation.cc:407] Connection header received
0811 22:13:03.396251 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.396254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.396318 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.396401 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:03.397157 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.398295 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.399865 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.400310 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.400414 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.404650 (+ 4236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.131:35163: BlockingRecv error: recv got EOF from 127.10.173.131:35163 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 22:13:03 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:03 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.129@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:03.451849 25609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.442544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.129:55809)
0811 22:13:03.443261 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:03.443277 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:03.443434 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:03.443803 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:03.443808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:03.444280 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:03.444453 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.444462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.445579 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.445589 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.446610 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.446623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.447471 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.447478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.447712 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.448458 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:03.448477 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:03.451489 (+ 3012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":351,"threads_started":1}
W20260811 22:13:03.452018 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:03.442728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:55809 (local address 127.10.173.190:45765)
0811 22:13:03.442986 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.442995 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.443364 (+ 369us) server_negotiation.cc:407] Connection header received
0811 22:13:03.443572 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.443582 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.443646 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.443790 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:03.444607 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.445416 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.446752 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.447285 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.447421 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.451712 (+ 4291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.129:55809: BlockingRecv error: recv got EOF from 127.10.173.129:55809 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 22:13:03 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:13:03 dist-test-slave-68xk krb5kdc[23896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.173.130@KRBTEST.COM for oryx/127.10.173.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:13:03.925527 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.916553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45765 (local address 127.10.173.130:41293)
0811 22:13:03.917156 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:03.917176 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:03.917307 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:03.917708 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:03.917712 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:03.918143 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:03.918323 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.918333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.919511 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.919516 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.920464 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.920477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.921168 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.921172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.921290 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.921887 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:03.921905 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:03.925122 (+ 3217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.173.190:45765: Server oryx/127.10.173.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":305,"threads_started":1}
W20260811 22:13:03.925587 25100 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:03.916651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:41293 (local address 127.10.173.190:45765)
0811 22:13:03.916926 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.916932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.917266 (+ 334us) server_negotiation.cc:407] Connection header received
0811 22:13:03.917458 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.917462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.917528 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.917618 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:03.918525 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.919365 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.920590 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.921027 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.921118 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.925348 (+ 4230us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.173.130:41293: BlockingRecv error: recv got EOF from 127.10.173.130:41293 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 22:13:04.219224 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 24845
I20260811 22:13:04.232162 25614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:04.220671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41328 (local address 127.10.173.129:35143)
0811 22:13:04.221254 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:04.221262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:04.221290 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:13:04.221350 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:04.221356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:04.221578 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:04.221728 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:04.223268 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:04.224341 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:04.225588 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:04.226216 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:04.226337 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:04.228963 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:04.228971 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:04.228985 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:04.229017 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:04.231632 (+ 2615us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.173.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 22:13:04.231916 (+ 284us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41328: Request ticket server kudu/127.10.173.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":282,"threads_started":1}
W20260811 22:13:04.232290 25614 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41328: Request ticket server kudu/127.10.173.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 22:13:04.232579 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 24845: Not authorized: Client connection negotiation failed: client connection to 127.10.173.129:35143: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.173.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 22:13:04.232745 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 24845
I20260811 22:13:04.261986 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 25101
I20260811 22:13:04.274720 25618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:04.263143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38410 (local address 127.10.173.130:38913)
0811 22:13:04.263839 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:04.263847 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:04.263880 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:13:04.263955 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:04.263962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:04.264252 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:04.264429 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:04.265756 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:04.267043 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:04.268240 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:04.268800 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:04.268904 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:04.271538 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:04.271548 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:04.271568 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:04.271605 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:04.274122 (+ 2517us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.173.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 22:13:04.274389 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38410: Request ticket server kudu/127.10.173.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":380,"threads_started":1}
W20260811 22:13:04.274838 25618 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38410: Request ticket server kudu/127.10.173.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 22:13:04.274871 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 25101: Not authorized: Client connection negotiation failed: client connection to 127.10.173.130:38913: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.173.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 22:13:04.275104 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 25101
I20260811 22:13:04.301426 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 25321
I20260811 22:13:04.314275 25619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:04.302517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53278 (local address 127.10.173.131:35673)
0811 22:13:04.303175 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:04.303183 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:04.303212 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:13:04.303275 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:04.303281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:04.303514 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:04.303675 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:04.305170 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:04.306401 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:04.307690 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:04.308282 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:04.308400 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:04.311094 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:04.311103 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:04.311117 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:04.311152 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:04.313684 (+ 2532us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.173.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 22:13:04.313953 (+ 269us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53278: Request ticket server kudu/127.10.173.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":302,"threads_started":1}
W20260811 22:13:04.314391 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 25321: Not authorized: Client connection negotiation failed: client connection to 127.10.173.131:35673: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.10.173.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 22:13:04.314394 25619 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53278: Request ticket server kudu/127.10.173.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 22:13:04.314706 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 25321
I20260811 22:13:04.340984 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 24734
I20260811 22:13:04.342329 24833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:04.462123 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 24734
2026-08-11T22:13:04Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21846 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T22:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:04Z Disabled control of system clock
I20260811 22:13:04.556385 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:37937
--webserver_interface=127.10.173.190
--webserver_port=0
--builtin_ntp_servers=127.10.173.148:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:37937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:13:04.913059 25624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:04.913383 25624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:04.913558 25624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:04.922598 25624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:04.922713 25624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:04.922806 25624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:04.922922 25624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:04.923017 25624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:04.933483 25624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:37937
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:04.935066 25624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:04.937052 25624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:04.950335 25632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:04.950335 25633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:04.950717 25635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:04.951970 25624 server_base.cc:1034] running on GCE node
I20260811 22:13:04.952864 25624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:04.954550 25624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:04.955802 25624 hybrid_clock.cc:648] HybridClock initialized: now 1786486384955735 us; error 70 us; skew 500 ppm
I20260811 22:13:04.956234 25624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:13:04.958911 25624 webserver.cc:460] Webserver started at http://127.10.173.190:44691/ using document root <none> and password file <none>
I20260811 22:13:04.959708 25624 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:04.959858 25624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:04.960234 25624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:04.963616 25624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "f7ba3ca25107442fab0369873968ff49"
format_stamp: "Formatted at 2026-08-11 22:13:04 on dist-test-slave-68xk"
server_key: "da81b4866359d8bfc0b98119966f94c4"
server_key_iv: "18b5c280c4400e7d4b459d9c43c9d312"
server_key_version: "encryptionkey@0"
I20260811 22:13:04.964510 25624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "f7ba3ca25107442fab0369873968ff49"
format_stamp: "Formatted at 2026-08-11 22:13:04 on dist-test-slave-68xk"
server_key: "da81b4866359d8bfc0b98119966f94c4"
server_key_iv: "18b5c280c4400e7d4b459d9c43c9d312"
server_key_version: "encryptionkey@0"
I20260811 22:13:04.970247 25624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:13:04.974264 25646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:04.975755 25624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:04.975986 25624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "f7ba3ca25107442fab0369873968ff49"
format_stamp: "Formatted at 2026-08-11 22:13:04 on dist-test-slave-68xk"
server_key: "da81b4866359d8bfc0b98119966f94c4"
server_key_iv: "18b5c280c4400e7d4b459d9c43c9d312"
server_key_version: "encryptionkey@0"
I20260811 22:13:04.976244 25624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:04.997812 25624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:04.998747 25624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:04.999069 25624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:05.021144 25624 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:37937
I20260811 22:13:05.021245 25725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:37937 every 8 connection(s)
I20260811 22:13:05.022718 25624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:05.023413 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 25624
I20260811 22:13:05.023687 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:13:05.024456 10934 external_mini_cluster.cc:1428] Setting key f0ab9eac4973f295ea93ab33bc45beee
I20260811 22:13:05.027941 25726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:05.041445 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:05.026535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50686 (local address 127.10.173.190:37937)
0811 22:13:05.027730 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:05.027747 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:05.027802 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:13:05.030432 (+ 2630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:05.030463 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:05.031066 (+ 603us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:05.031572 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:05.032176 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:05.034522 (+ 2346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:05.035825 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:05.036958 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:05.037175 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:05.038317 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:05.038333 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:05.038352 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:05.039189 (+ 837us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:05.039293 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:05.039320 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:05.039474 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:05.039645 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:13:05.039996 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":345,"threads_started":1}
I20260811 22:13:05.043691 25726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49: Bootstrap starting.
I20260811 22:13:05.047487 25726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:05.048807 25726 log.cc:824] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:05.051765 25726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49: No bootstrap required, opened a new log
I20260811 22:13:05.059152 25726 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ba3ca25107442fab0369873968ff49" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37937 } }
I20260811 22:13:05.059552 25726 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:05.059679 25726 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ba3ca25107442fab0369873968ff49, State: Initialized, Role: FOLLOWER
I20260811 22:13:05.060322 25726 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ba3ca25107442fab0369873968ff49" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37937 } }
I20260811 22:13:05.060627 25726 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:05.060775 25726 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:05.060912 25726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:05.063056 25726 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ba3ca25107442fab0369873968ff49" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37937 } }
I20260811 22:13:05.063503 25726 leader_election.cc:302] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7ba3ca25107442fab0369873968ff49; no voters:
I20260811 22:13:05.064059 25726 leader_election.cc:288] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:05.064291 25734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:05.065258 25734 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [term 1 LEADER]: Becoming Leader. State: Replica: f7ba3ca25107442fab0369873968ff49, State: Running, Role: LEADER
I20260811 22:13:05.065861 25734 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ba3ca25107442fab0369873968ff49" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37937 } }
I20260811 22:13:05.066474 25726 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:05.068861 25736 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7ba3ca25107442fab0369873968ff49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ba3ca25107442fab0369873968ff49" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37937 } } }
I20260811 22:13:05.069267 25736 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:05.069999 25737 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7ba3ca25107442fab0369873968ff49. Latest consensus state: current_term: 1 leader_uuid: "f7ba3ca25107442fab0369873968ff49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7ba3ca25107442fab0369873968ff49" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 37937 } } }
I20260811 22:13:05.070390 25737 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:05.072388 25751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:05.078066 25751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:05.088552 25751 catalog_manager.cc:1347] Generated new cluster ID: cc44f235545f4ebdb39939cd84359bcb
I20260811 22:13:05.088779 25751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:05.104696 25751 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:05.106918 25751 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:05.115355 25751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7ba3ca25107442fab0369873968ff49: Generated new TSK 0
I20260811 22:13:05.116178 25751 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:13:05.125500 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37937
--builtin_ntp_servers=127.10.173.148:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 22:13:05.500488 25765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:05.500797 25765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:05.500885 25765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:05.508242 25765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:05.508374 25765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:05.508484 25765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:13:05.517661 25765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:05.519043 25765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:05.520895 25765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:05.534536 25775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:05.534629 25776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:05.534591 25779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:05.535362 25765 server_base.cc:1034] running on GCE node
I20260811 22:13:05.535924 25765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:05.536799 25765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:05.538008 25765 hybrid_clock.cc:648] HybridClock initialized: now 1786486385537966 us; error 31 us; skew 500 ppm
I20260811 22:13:05.538358 25765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:13:05.540948 25765 webserver.cc:460] Webserver started at http://127.10.173.129:44395/ using document root <none> and password file <none>
I20260811 22:13:05.541658 25765 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:05.541782 25765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:05.542207 25765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:05.545116 25765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "f2593cb347994e238b1acfa802d46c9d"
format_stamp: "Formatted at 2026-08-11 22:13:05 on dist-test-slave-68xk"
server_key: "6733ff24d080bb789d41220b2578b651"
server_key_iv: "72a02e99240c9ec0a63aa31151fb2100"
server_key_version: "encryptionkey@0"
I20260811 22:13:05.545900 25765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "f2593cb347994e238b1acfa802d46c9d"
format_stamp: "Formatted at 2026-08-11 22:13:05 on dist-test-slave-68xk"
server_key: "6733ff24d080bb789d41220b2578b651"
server_key_iv: "72a02e99240c9ec0a63aa31151fb2100"
server_key_version: "encryptionkey@0"
I20260811 22:13:05.551270 25765 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 22:13:05.554739 25785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:05.556140 25765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:13:05.556330 25765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "f2593cb347994e238b1acfa802d46c9d"
format_stamp: "Formatted at 2026-08-11 22:13:05 on dist-test-slave-68xk"
server_key: "6733ff24d080bb789d41220b2578b651"
server_key_iv: "72a02e99240c9ec0a63aa31151fb2100"
server_key_version: "encryptionkey@0"
I20260811 22:13:05.556545 25765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:05.585201 25765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:05.586129 25765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:05.586548 25765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:05.587605 25765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:05.589135 25765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:05.589285 25765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:05.589425 25765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:05.589521 25765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:05.624356 25765 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:36391
I20260811 22:13:05.624452 25974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:36391 every 8 connection(s)
I20260811 22:13:05.625703 25765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:05.632706 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 25765
I20260811 22:13:05.632934 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:13:05.633870 10934 external_mini_cluster.cc:1428] Setting key 4d19d50efaaa9152b76b08210f529c7b
I20260811 22:13:05.637082 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37937
--builtin_ntp_servers=127.10.173.148:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:13:05.639638 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:05.628275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:57515 (local address 127.10.173.190:37937)
0811 22:13:05.628752 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:05.628756 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:05.629899 (+ 1143us) server_negotiation.cc:407] Connection header received
0811 22:13:05.631762 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:05.631768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:05.631858 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:05.631981 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:05.633460 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:05.634577 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:05.636578 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:05.636973 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:05.637084 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:05.638782 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:05.638787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:05.638794 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:05.638812 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:05.638853 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:05.638866 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:05.638951 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:05.639252 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 22:13:05.639349 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":260,"threads_started":1}
I20260811 22:13:05.641122 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:05.628741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37937 (local address 127.10.173.129:57515)
0811 22:13:05.629717 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:05.629754 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:05.631479 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:05.632271 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:05.632284 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:05.632413 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:05.633265 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:05.633291 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:05.634779 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:05.634783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:05.636406 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:05.636422 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:05.637229 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:05.637239 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:05.637569 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:05.638372 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:05.638377 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:05.638448 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:05.638559 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:05.638975 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:05.638989 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:05.638999 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:13:05.639199 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 22:13:05.639744 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":325,"threads_started":1}
I20260811 22:13:05.642858 25975 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37937
I20260811 22:13:05.643190 25975 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:05.644003 25975 heartbeater.cc:511] Master 127.10.173.190:37937 requested a full tablet report, sending...
I20260811 22:13:05.646363 25679 ts_manager.cc:194] Registered new tserver with Master: f2593cb347994e238b1acfa802d46c9d (127.10.173.129:36391)
I20260811 22:13:05.647910 25679 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.129:57515
W20260811 22:13:05.985335 25981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:05.985671 25981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:05.985805 25981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:05.992874 25981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:05.992971 25981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:05.993080 25981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:13:06.002194 25981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:06.003566 25981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:06.005508 25981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:06.018920 25994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:06.018998 25989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:06.019030 25990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:06.020145 25981 server_base.cc:1034] running on GCE node
I20260811 22:13:06.020885 25981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:06.021895 25981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:06.023166 25981 hybrid_clock.cc:648] HybridClock initialized: now 1786486386023077 us; error 80 us; skew 500 ppm
I20260811 22:13:06.023633 25981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:13:06.026110 25981 webserver.cc:460] Webserver started at http://127.10.173.130:36949/ using document root <none> and password file <none>
I20260811 22:13:06.026883 25981 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:06.026991 25981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:06.027349 25981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:06.030169 25981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "c5d56d08d60f46d798ed5b7130c9b6e9"
format_stamp: "Formatted at 2026-08-11 22:13:06 on dist-test-slave-68xk"
server_key: "2f4e1e16816ac22db9573e5c7ac2dcb9"
server_key_iv: "f413b345ae7b58684050d312e68662e9"
server_key_version: "encryptionkey@0"
I20260811 22:13:06.031019 25981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "c5d56d08d60f46d798ed5b7130c9b6e9"
format_stamp: "Formatted at 2026-08-11 22:13:06 on dist-test-slave-68xk"
server_key: "2f4e1e16816ac22db9573e5c7ac2dcb9"
server_key_iv: "f413b345ae7b58684050d312e68662e9"
server_key_version: "encryptionkey@0"
I20260811 22:13:06.036142 25981 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:13:06.039703 26004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:06.041100 25981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:13:06.041311 25981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "c5d56d08d60f46d798ed5b7130c9b6e9"
format_stamp: "Formatted at 2026-08-11 22:13:06 on dist-test-slave-68xk"
server_key: "2f4e1e16816ac22db9573e5c7ac2dcb9"
server_key_iv: "f413b345ae7b58684050d312e68662e9"
server_key_version: "encryptionkey@0"
I20260811 22:13:06.041543 25981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:06.055883 25981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:06.056777 25981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:06.057113 25981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:06.058151 25981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:06.059851 25981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:06.059947 25981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:06.060086 25981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:06.060163 25981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:06.096606 25981 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:42181
I20260811 22:13:06.096647 26199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:42181 every 8 connection(s)
I20260811 22:13:06.098114 25981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:06.108321 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 25981
I20260811 22:13:06.108552 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:13:06.109210 10934 external_mini_cluster.cc:1428] Setting key 0564343cab40e807937d147650e8f693
I20260811 22:13:06.111385 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:06.100940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:56535 (local address 127.10.173.190:37937)
0811 22:13:06.101190 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:06.101197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:06.102599 (+ 1402us) server_negotiation.cc:407] Connection header received
0811 22:13:06.104320 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:06.104324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:06.104396 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:06.104493 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:06.105712 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.106506 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.108063 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.108849 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.109010 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:06.110544 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:06.110550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:06.110555 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:06.110574 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:06.110615 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:06.110631 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:06.110724 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:06.110960 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 22:13:06.111054 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:13:06.112843 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:06.101443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37937 (local address 127.10.173.130:56535)
0811 22:13:06.102367 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:06.102402 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:06.104041 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:06.104754 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:06.104762 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:06.104869 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:06.105526 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:06.105543 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.106666 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.106670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:06.107894 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:06.107906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.109069 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.109075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:06.109276 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:06.110107 (+ 831us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:06.110112 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:06.110180 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:06.110292 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:06.110747 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:06.110757 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:06.110764 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:13:06.110929 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 22:13:06.111342 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":279,"threads_started":1}
I20260811 22:13:06.112946 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37937
--builtin_ntp_servers=127.10.173.148:35223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:13:06.114387 26200 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37937
I20260811 22:13:06.114765 26200 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:06.115525 26200 heartbeater.cc:511] Master 127.10.173.190:37937 requested a full tablet report, sending...
I20260811 22:13:06.116950 25679 ts_manager.cc:194] Registered new tserver with Master: c5d56d08d60f46d798ed5b7130c9b6e9 (127.10.173.130:42181)
I20260811 22:13:06.117739 25679 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.130:56535
W20260811 22:13:06.449410 26208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:06.449694 26208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:06.449810 26208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:06.456965 26208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:06.457067 26208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:06.457255 26208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:13:06.466259 26208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:37937
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:06.467617 26208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:06.469411 26208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:06.482532 26214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:06.482587 26215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:06.482543 26218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:06.483722 26208 server_base.cc:1034] running on GCE node
I20260811 22:13:06.484373 26208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:06.485338 26208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:06.486558 26208 hybrid_clock.cc:648] HybridClock initialized: now 1786486386486514 us; error 56 us; skew 500 ppm
I20260811 22:13:06.486901 26208 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:13:06.489102 26208 webserver.cc:460] Webserver started at http://127.10.173.131:41189/ using document root <none> and password file <none>
I20260811 22:13:06.489822 26208 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:06.489955 26208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:06.490280 26208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:06.493091 26208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "025f7e50555741cc8338ca8e57b6288c"
format_stamp: "Formatted at 2026-08-11 22:13:06 on dist-test-slave-68xk"
server_key: "a41b41b00e101f206420d2c499d0f2ce"
server_key_iv: "fd93450a4abbecd87d7cbb25dc1d8055"
server_key_version: "encryptionkey@0"
I20260811 22:13:06.493911 26208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "025f7e50555741cc8338ca8e57b6288c"
format_stamp: "Formatted at 2026-08-11 22:13:06 on dist-test-slave-68xk"
server_key: "a41b41b00e101f206420d2c499d0f2ce"
server_key_iv: "fd93450a4abbecd87d7cbb25dc1d8055"
server_key_version: "encryptionkey@0"
I20260811 22:13:06.499155 26208 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:13:06.502884 26232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:06.504261 26208 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:13:06.504456 26208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "025f7e50555741cc8338ca8e57b6288c"
format_stamp: "Formatted at 2026-08-11 22:13:06 on dist-test-slave-68xk"
server_key: "a41b41b00e101f206420d2c499d0f2ce"
server_key_iv: "fd93450a4abbecd87d7cbb25dc1d8055"
server_key_version: "encryptionkey@0"
I20260811 22:13:06.504712 26208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:06.527448 26208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:06.528290 26208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:06.528599 26208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:06.529553 26208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:06.531037 26208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:06.531175 26208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:06.531298 26208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:06.531385 26208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:06.565244 26208 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:33899
I20260811 22:13:06.565332 26414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:33899 every 8 connection(s)
I20260811 22:13:06.566605 26208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:06.572117 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 26208
I20260811 22:13:06.572319 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:13:06.572781 10934 external_mini_cluster.cc:1428] Setting key 8e316b9a243a350a4e0af8eeb3fad8e4
I20260811 22:13:06.579871 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:06.569288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:34429 (local address 127.10.173.190:37937)
0811 22:13:06.569501 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:06.569507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:06.570952 (+ 1445us) server_negotiation.cc:407] Connection header received
0811 22:13:06.572894 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:06.572901 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:06.572988 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:06.573110 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:06.574442 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.575418 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.577244 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.577638 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.577739 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:06.579102 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:06.579107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:06.579110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:06.579127 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:06.579159 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:06.579170 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:06.579248 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:06.579541 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 22:13:06.579620 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:13:06.580960 26417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:06.569843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:37937 (local address 127.10.173.131:34429)
0811 22:13:06.570795 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:06.570838 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:06.572626 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:06.573337 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:06.573344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:06.573428 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:06.574250 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:06.574280 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.575604 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.575609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:06.577105 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:06.577120 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.577845 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.577850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:06.578059 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:06.578804 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:06.578807 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:06.578847 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:06.578930 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:06.579282 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:06.579291 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:06.579298 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:13:06.579541 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 22:13:06.579851 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":296,"threads_started":1}
I20260811 22:13:06.582264 26415 heartbeater.cc:347] Connected to a master server at 127.10.173.190:37937
I20260811 22:13:06.582602 26415 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:06.583349 26415 heartbeater.cc:511] Master 127.10.173.190:37937 requested a full tablet report, sending...
I20260811 22:13:06.584730 25679 ts_manager.cc:194] Registered new tserver with Master: 025f7e50555741cc8338ca8e57b6288c (127.10.173.131:33899)
I20260811 22:13:06.585513 25679 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.131:34429
I20260811 22:13:06.588526 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:13:06.594751 25979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:06.593233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.10.173.190:37937)
0811 22:13:06.593433 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:06.593438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:06.593857 (+ 419us) server_negotiation.cc:407] Connection header received
0811 22:13:06.593992 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:06.593996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:06.594057 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:06.594138 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:06.594559 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50692: BlockingRecv error: recv got EOF from 127.0.0.1:50692 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:13:06.595714 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 25765
I20260811 22:13:06.603442 26431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:06.596644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54086 (local address 127.10.173.129:36391)
0811 22:13:06.597237 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:06.597249 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:06.597277 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:13:06.597343 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:06.597348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:06.597552 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:06.597684 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:06.598020 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.599094 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.600390 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.600946 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.601044 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:06.602054 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:06.602062 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:06.602074 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:06.602763 (+ 689us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:06.602808 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:06.602829 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:06.602933 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:06.603027 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 22:13:06.603143 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":323,"threads_started":1}
I20260811 22:13:06.604682 25966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:06.687132 25975 heartbeater.cc:503] Master 127.10.173.190:37937 was elected leader, sending a full tablet report...
I20260811 22:13:06.688228 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 25765
W20260811 22:13:06.710801 25652 connection.cc:535] server connection from 127.10.173.129:57515 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:13:06.711135 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 25981
I20260811 22:13:06.718986 26435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:06.712038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40982 (local address 127.10.173.130:42181)
0811 22:13:06.712679 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:06.712686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:06.712715 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:13:06.712785 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:06.712792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:06.713017 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:06.713163 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:06.713513 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.714649 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.715869 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.716433 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.716530 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:06.717531 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:06.717539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:06.717552 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:06.718181 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:06.718227 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:06.718246 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:06.718342 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:06.718515 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 22:13:06.718682 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":357,"threads_started":1}
I20260811 22:13:06.720288 26188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:06.805454 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 25981
I20260811 22:13:06.832733 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 26208
I20260811 22:13:06.842034 26442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:06.833776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.10.173.131:33899)
0811 22:13:06.834581 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:06.834589 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:06.834628 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:13:06.834696 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:06.834703 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:06.834938 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:06.835140 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:06.835711 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.836890 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.838340 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:06.839089 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:06.839243 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:06.840502 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:06.840511 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:06.840526 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:06.841138 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:06.841187 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:06.841208 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:06.841339 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:06.841513 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:13:06.841681 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":364,"threads_started":1}
I20260811 22:13:06.843420 26408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:06.926762 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 26208
I20260811 22:13:06.949229 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 25624
I20260811 22:13:06.950569 25721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:07.056985 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 25624
2026-08-11T22:13:07Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2588 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T22:13:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:07Z Disabled control of system clock
I20260811 22:13:07.147164 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41741
--webserver_interface=127.10.173.190
--webserver_port=0
--builtin_ntp_servers=127.10.173.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:41741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:13:07.493432 26452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:07.493777 26452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:07.493935 26452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:07.501314 26452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:07.501482 26452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:07.501564 26452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:07.501657 26452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:07.501736 26452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:07.511096 26452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:41741
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:07.512542 26452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:07.514462 26452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:07.526948 26467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:07.527009 26463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:07.527231 26461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:07.527968 26452 server_base.cc:1034] running on GCE node
I20260811 22:13:07.528780 26452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:07.530225 26452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:07.531474 26452 hybrid_clock.cc:648] HybridClock initialized: now 1786486387531430 us; error 51 us; skew 500 ppm
I20260811 22:13:07.531860 26452 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:13:07.539194 26452 webserver.cc:460] Webserver started at http://127.10.173.190:34883/ using document root <none> and password file <none>
I20260811 22:13:07.539947 26452 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:07.540079 26452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:07.540417 26452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:07.543550 26452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "c7ae5e337f904e4f9d7390536f963a06"
format_stamp: "Formatted at 2026-08-11 22:13:07 on dist-test-slave-68xk"
server_key: "b83e683e6b003b04cd0b65a540986cab"
server_key_iv: "fd921e14324df9020790a091d8ddaebb"
server_key_version: "encryptionkey@0"
I20260811 22:13:07.544364 26452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "c7ae5e337f904e4f9d7390536f963a06"
format_stamp: "Formatted at 2026-08-11 22:13:07 on dist-test-slave-68xk"
server_key: "b83e683e6b003b04cd0b65a540986cab"
server_key_iv: "fd921e14324df9020790a091d8ddaebb"
server_key_version: "encryptionkey@0"
I20260811 22:13:07.549597 26452 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:13:07.553221 26476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:07.554773 26452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:07.554970 26452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "c7ae5e337f904e4f9d7390536f963a06"
format_stamp: "Formatted at 2026-08-11 22:13:07 on dist-test-slave-68xk"
server_key: "b83e683e6b003b04cd0b65a540986cab"
server_key_iv: "fd921e14324df9020790a091d8ddaebb"
server_key_version: "encryptionkey@0"
I20260811 22:13:07.555198 26452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:07.561259 26452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:07.562039 26452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:07.562352 26452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:07.582093 26452 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:41741
I20260811 22:13:07.582188 26551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:41741 every 8 connection(s)
I20260811 22:13:07.583556 26452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:07.587889 26553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:07.593894 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 26452
I20260811 22:13:07.594110 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:13:07.595081 10934 external_mini_cluster.cc:1428] Setting key 92144214412a112ee7214f8f6ab24681
I20260811 22:13:07.600997 26553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06: Bootstrap starting.
I20260811 22:13:07.604274 26557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:07.597581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37838 (local address 127.10.173.190:41741)
0811 22:13:07.598800 (+ 1219us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:07.598814 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:07.598859 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:13:07.600522 (+ 1663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:07.600546 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:07.600976 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:07.601411 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:07.602607 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:07.602613 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:07.602629 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:07.602644 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:07.602712 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:07.602730 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:07.602985 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:07.603156 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 22:13:07.603403 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":392,"threads_started":1}
I20260811 22:13:07.604913 26553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:07.606315 26553 log.cc:824] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:07.609361 26553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06: No bootstrap required, opened a new log
I20260811 22:13:07.616919 26553 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ae5e337f904e4f9d7390536f963a06" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41741 } }
I20260811 22:13:07.617286 26553 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:07.617393 26553 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ae5e337f904e4f9d7390536f963a06, State: Initialized, Role: FOLLOWER
I20260811 22:13:07.618032 26553 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ae5e337f904e4f9d7390536f963a06" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41741 } }
I20260811 22:13:07.618269 26553 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:07.618402 26553 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:07.618614 26553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:07.620769 26553 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ae5e337f904e4f9d7390536f963a06" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41741 } }
I20260811 22:13:07.621274 26553 leader_election.cc:302] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7ae5e337f904e4f9d7390536f963a06; no voters:
I20260811 22:13:07.621872 26553 leader_election.cc:288] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:07.622020 26559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:07.622818 26559 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [term 1 LEADER]: Becoming Leader. State: Replica: c7ae5e337f904e4f9d7390536f963a06, State: Running, Role: LEADER
I20260811 22:13:07.623395 26559 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ae5e337f904e4f9d7390536f963a06" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41741 } }
I20260811 22:13:07.624293 26553 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:07.626467 26562 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7ae5e337f904e4f9d7390536f963a06. Latest consensus state: current_term: 1 leader_uuid: "c7ae5e337f904e4f9d7390536f963a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ae5e337f904e4f9d7390536f963a06" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41741 } } }
I20260811 22:13:07.626489 26560 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7ae5e337f904e4f9d7390536f963a06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7ae5e337f904e4f9d7390536f963a06" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41741 } } }
I20260811 22:13:07.626919 26562 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:07.626916 26560 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:07.628546 26568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:07.633821 26568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:07.642920 26568 catalog_manager.cc:1347] Generated new cluster ID: 27c9207452ae4961b63ff80aec3682b8
I20260811 22:13:07.643035 26568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:07.684739 26568 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:07.685242 26568 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:07.690658 26568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7ae5e337f904e4f9d7390536f963a06: Generated new TSK 0
I20260811 22:13:07.691490 26568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:13:07.703118 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41741
--builtin_ntp_servers=127.10.173.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:13:08.077728 26587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:08.077998 26587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:08.078150 26587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:08.085369 26587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:08.085489 26587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:08.085726 26587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:13:08.094750 26587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:08.096101 26587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:08.097903 26587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:08.111123 26605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:08.111019 26600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:08.111403 26599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:08.112165 26587 server_base.cc:1034] running on GCE node
I20260811 22:13:08.112700 26587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:08.113570 26587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:08.114773 26587 hybrid_clock.cc:648] HybridClock initialized: now 1786486388114708 us; error 53 us; skew 500 ppm
I20260811 22:13:08.115093 26587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:13:08.117457 26587 webserver.cc:460] Webserver started at http://127.10.173.129:46693/ using document root <none> and password file <none>
I20260811 22:13:08.118147 26587 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:08.118283 26587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:08.118660 26587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:08.121438 26587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "9b66c80c057948908bd085da9184865f"
format_stamp: "Formatted at 2026-08-11 22:13:08 on dist-test-slave-68xk"
server_key: "be07b50ccdfe485a39714e7459ed0623"
server_key_iv: "2b0f326615742a9ae41e9452452e75ed"
server_key_version: "encryptionkey@0"
I20260811 22:13:08.122226 26587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "9b66c80c057948908bd085da9184865f"
format_stamp: "Formatted at 2026-08-11 22:13:08 on dist-test-slave-68xk"
server_key: "be07b50ccdfe485a39714e7459ed0623"
server_key_iv: "2b0f326615742a9ae41e9452452e75ed"
server_key_version: "encryptionkey@0"
I20260811 22:13:08.127410 26587 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:13:08.130976 26611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:08.132263 26587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:13:08.132437 26587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "9b66c80c057948908bd085da9184865f"
format_stamp: "Formatted at 2026-08-11 22:13:08 on dist-test-slave-68xk"
server_key: "be07b50ccdfe485a39714e7459ed0623"
server_key_iv: "2b0f326615742a9ae41e9452452e75ed"
server_key_version: "encryptionkey@0"
I20260811 22:13:08.132675 26587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:08.138253 26587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:08.139104 26587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:08.139379 26587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:08.140304 26587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:08.141671 26587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:08.141795 26587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:08.141929 26587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:08.142016 26587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:08.173171 26587 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:45969
I20260811 22:13:08.173229 26792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:45969 every 8 connection(s)
I20260811 22:13:08.174527 26587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:08.179930 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 26587
I20260811 22:13:08.180138 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:13:08.180792 10934 external_mini_cluster.cc:1428] Setting key 942d9f26e7d46270135b645e73c72c09
I20260811 22:13:08.183034 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:08.177209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:60425 (local address 127.10.173.190:41741)
0811 22:13:08.177710 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:08.177715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:08.178815 (+ 1100us) server_negotiation.cc:407] Connection header received
0811 22:13:08.180641 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:08.180647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:08.180734 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:08.180859 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:08.182201 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:08.182207 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:08.182212 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:08.182232 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:08.182284 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:08.182298 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:08.182446 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:08.182673 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 22:13:08.182739 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":265,"threads_started":1}
I20260811 22:13:08.183928 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41741
--builtin_ntp_servers=127.10.173.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:13:08.184212 26797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:08.177671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41741 (local address 127.10.173.129:60425)
0811 22:13:08.178654 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:08.178691 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:08.180363 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:08.181110 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:08.181118 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:08.181207 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:08.181908 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:08.181911 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:08.181945 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:08.182027 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:08.182397 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:08.182405 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:08.182446 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 22:13:08.182724 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 22:13:08.182974 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":296,"threads_started":1}
I20260811 22:13:08.187853 26793 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41741
I20260811 22:13:08.188184 26793 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:08.188876 26793 heartbeater.cc:511] Master 127.10.173.190:41741 requested a full tablet report, sending...
I20260811 22:13:08.191251 26498 ts_manager.cc:194] Registered new tserver with Master: 9b66c80c057948908bd085da9184865f (127.10.173.129:45969)
W20260811 22:13:08.526775 26798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:08.527113 26798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:08.527267 26798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:08.534821 26798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:08.534951 26798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:08.535126 26798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:13:08.544435 26798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:08.545769 26798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:08.547681 26798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:08.561305 26808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:08.561288 26806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:08.561510 26804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:08.562292 26798 server_base.cc:1034] running on GCE node
I20260811 22:13:08.562904 26798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:08.563823 26798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:08.565061 26798 hybrid_clock.cc:648] HybridClock initialized: now 1786486388564997 us; error 46 us; skew 500 ppm
I20260811 22:13:08.565438 26798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:13:08.568030 26798 webserver.cc:460] Webserver started at http://127.10.173.130:33111/ using document root <none> and password file <none>
I20260811 22:13:08.568789 26798 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:08.568913 26798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:08.569316 26798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:08.572513 26798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "41070de0480b46b09f306d6c575f4465"
format_stamp: "Formatted at 2026-08-11 22:13:08 on dist-test-slave-68xk"
server_key: "07cdd3e88272883c39ba5585e8459002"
server_key_iv: "61288f9adbb7247c9003cfccc900d135"
server_key_version: "encryptionkey@0"
I20260811 22:13:08.573367 26798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "41070de0480b46b09f306d6c575f4465"
format_stamp: "Formatted at 2026-08-11 22:13:08 on dist-test-slave-68xk"
server_key: "07cdd3e88272883c39ba5585e8459002"
server_key_iv: "61288f9adbb7247c9003cfccc900d135"
server_key_version: "encryptionkey@0"
I20260811 22:13:08.578809 26798 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:13:08.582641 26817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:08.584277 26798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:13:08.584447 26798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "41070de0480b46b09f306d6c575f4465"
format_stamp: "Formatted at 2026-08-11 22:13:08 on dist-test-slave-68xk"
server_key: "07cdd3e88272883c39ba5585e8459002"
server_key_iv: "61288f9adbb7247c9003cfccc900d135"
server_key_version: "encryptionkey@0"
I20260811 22:13:08.584689 26798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:08.592051 26798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:08.592952 26798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:08.593219 26798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:08.594237 26798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:08.596016 26798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:08.596127 26798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:08.596240 26798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:08.596311 26798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:08.630453 26798 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:35313
I20260811 22:13:08.630553 26996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:35313 every 8 connection(s)
I20260811 22:13:08.631788 26798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:08.632803 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 26798
I20260811 22:13:08.632995 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:13:08.633499 10934 external_mini_cluster.cc:1428] Setting key 2de7f9c2a858a21613907fafc26fba28
I20260811 22:13:08.636878 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41741
--builtin_ntp_servers=127.10.173.148:34789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:13:08.641024 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:08.634486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:53341 (local address 127.10.173.190:41741)
0811 22:13:08.634728 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:08.634734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:08.636959 (+ 2225us) server_negotiation.cc:407] Connection header received
0811 22:13:08.638720 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:08.638729 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:08.638801 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:08.638913 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:08.640219 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:08.640227 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:08.640235 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:08.640258 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:08.640303 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:08.640321 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:08.640458 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:08.640612 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 22:13:08.640681 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:13:08.641867 27001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:08.635172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41741 (local address 127.10.173.130:53341)
0811 22:13:08.636775 (+ 1603us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:08.636829 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:08.638455 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:08.639128 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:08.639136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:08.639221 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:08.639917 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:08.639920 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:08.639955 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:08.640049 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:08.640422 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:08.640429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:08.640435 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:13:08.640646 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 22:13:08.640842 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"thread_start_us":818,"threads_started":1}
I20260811 22:13:08.643211 26997 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41741
I20260811 22:13:08.643504 26997 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:08.644080 26997 heartbeater.cc:511] Master 127.10.173.190:41741 requested a full tablet report, sending...
I20260811 22:13:08.645380 26498 ts_manager.cc:194] Registered new tserver with Master: 41070de0480b46b09f306d6c575f4465 (127.10.173.130:35313)
W20260811 22:13:08.982316 27002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:08.982664 27002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:08.982839 27002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:08.990163 27002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:08.990264 27002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:08.990506 27002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:13:08.999517 27002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:09.000880 27002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:09.002719 27002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:09.023761 27017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:09.023880 27011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:09.023941 27012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:09.024605 27002 server_base.cc:1034] running on GCE node
I20260811 22:13:09.025282 27002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:09.026341 27002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:09.027627 27002 hybrid_clock.cc:648] HybridClock initialized: now 1786486389027565 us; error 41 us; skew 500 ppm
I20260811 22:13:09.027981 27002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:13:09.030638 27002 webserver.cc:460] Webserver started at http://127.10.173.131:41581/ using document root <none> and password file <none>
I20260811 22:13:09.031366 27002 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:09.031481 27002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:09.031833 27002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:09.034945 27002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "1033eff241114107b5659a1fd5bc84a4"
format_stamp: "Formatted at 2026-08-11 22:13:09 on dist-test-slave-68xk"
server_key: "b32fedc4a5ce8c2db67287254f59ea53"
server_key_iv: "c1fc2be1bdae0566bf6c7f48e037690a"
server_key_version: "encryptionkey@0"
I20260811 22:13:09.035845 27002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "1033eff241114107b5659a1fd5bc84a4"
format_stamp: "Formatted at 2026-08-11 22:13:09 on dist-test-slave-68xk"
server_key: "b32fedc4a5ce8c2db67287254f59ea53"
server_key_iv: "c1fc2be1bdae0566bf6c7f48e037690a"
server_key_version: "encryptionkey@0"
I20260811 22:13:09.041471 27002 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:13:09.045123 27027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:09.046567 27002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:09.046746 27002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "1033eff241114107b5659a1fd5bc84a4"
format_stamp: "Formatted at 2026-08-11 22:13:09 on dist-test-slave-68xk"
server_key: "b32fedc4a5ce8c2db67287254f59ea53"
server_key_iv: "c1fc2be1bdae0566bf6c7f48e037690a"
server_key_version: "encryptionkey@0"
I20260811 22:13:09.046974 27002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:09.052610 27002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:09.053424 27002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:09.053706 27002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:09.054692 27002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:09.056116 27002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:09.056205 27002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:09.056337 27002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:09.056453 27002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:09.091750 27002 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:35287
I20260811 22:13:09.091850 27212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:35287 every 8 connection(s)
I20260811 22:13:09.093127 27002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:09.095468 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 27002
I20260811 22:13:09.095638 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:13:09.096160 10934 external_mini_cluster.cc:1428] Setting key 9905c7ee8fe4a6079c58ad0f6573c079
I20260811 22:13:09.101924 26795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:09.095820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:50821 (local address 127.10.173.190:41741)
0811 22:13:09.096048 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:09.096056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:09.097731 (+ 1675us) server_negotiation.cc:407] Connection header received
0811 22:13:09.099874 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:09.099878 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:09.099934 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:09.100021 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:09.101285 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:09.101290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:09.101293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:09.101309 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:09.101341 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:09.101353 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:09.101461 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:09.101661 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 22:13:09.101717 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:13:09.102823 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:09.096445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41741 (local address 127.10.173.131:50821)
0811 22:13:09.097542 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:09.097593 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:09.099632 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:09.100211 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:09.100219 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:09.100305 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:09.100998 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:09.101001 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:09.101036 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:09.101122 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:09.101442 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:09.101450 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:09.101455 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:13:09.101679 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 22:13:09.101864 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":341,"threads_started":1}
I20260811 22:13:09.104102 27213 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41741
I20260811 22:13:09.104408 27213 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:09.105038 27213 heartbeater.cc:511] Master 127.10.173.190:41741 requested a full tablet report, sending...
I20260811 22:13:09.106351 26498 ts_manager.cc:194] Registered new tserver with Master: 1033eff241114107b5659a1fd5bc84a4 (127.10.173.131:35287)
I20260811 22:13:09.111826 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:13:09.118062 26795 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:09.116557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37854 (local address 127.10.173.190:41741)
0811 22:13:09.116734 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:09.116739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:09.117283 (+ 544us) server_negotiation.cc:407] Connection header received
0811 22:13:09.117408 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:09.117412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:09.117462 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:09.117551 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:09.117849 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37854: BlockingRecv error: recv got EOF from 127.0.0.1:37854 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:13:09.118854 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 26587
I20260811 22:13:09.122462 27230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:09.119786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.10.173.129:45969)
0811 22:13:09.120289 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:09.120295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:09.120321 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:13:09.120376 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:09.120382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:09.120573 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:09.120700 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:09.121803 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:09.121810 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:09.121820 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:09.121833 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:09.121878 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:09.121895 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:09.121996 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:09.122073 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 22:13:09.122198 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":234,"threads_started":1}
I20260811 22:13:09.123606 26786 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:09.204392 26793 heartbeater.cc:503] Master 127.10.173.190:41741 was elected leader, sending a full tablet report...
I20260811 22:13:09.206260 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 26587
I20260811 22:13:09.230041 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 26798
I20260811 22:13:09.233697 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:09.231078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.10.173.130:35313)
0811 22:13:09.231599 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:09.231607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:09.231634 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:13:09.231696 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:09.231702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:09.231915 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:09.232039 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:09.232973 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:09.232980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:09.232991 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:09.233011 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:09.233054 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:09.233068 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:09.233211 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:09.233290 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 22:13:09.233401 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":221,"threads_started":1}
I20260811 22:13:09.235030 26989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:09.315939 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 26798
I20260811 22:13:09.340377 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 27002
I20260811 22:13:09.345330 27235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:09.341587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42622 (local address 127.10.173.131:35287)
0811 22:13:09.342560 (+ 973us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:09.342572 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:09.342610 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:13:09.342680 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:09.342688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:09.342936 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:09.343097 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:09.344287 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:09.344295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:09.344306 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:09.344325 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:09.344367 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:09.344382 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:09.344526 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:09.344776 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 22:13:09.344891 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":531,"threads_started":1}
I20260811 22:13:09.346803 27206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:09.429653 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 27002
I20260811 22:13:09.452185 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 26452
I20260811 22:13:09.453505 26544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:09.556892 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 26452
2026-08-11T22:13:09Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2498 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:09 dist-test-slave-68xk krb5kdc[27241](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:09 dist-test-slave-68xk krb5kdc[27241](info): set up 2 sockets
Aug 11 22:13:09 dist-test-slave-68xk krb5kdc[27241](info): commencing operation
krb5kdc: starting...
W20260811 22:13:11.002667 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.399s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:13:11 dist-test-slave-68xk krb5kdc[27241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:13:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:13:11.294762 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:46583
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:46583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:11.630908 27258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:11.631199 27258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:11.631359 27258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:11.638670 27258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:11.638770 27258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:11.638898 27258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:11.638998 27258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:11.639072 27258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:11.648818 27258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:46583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:46583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:11.650120 27258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:11.651443 27258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:11.663443 27266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:11.663403 27267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:11.663370 27270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:11.664529 27258 server_base.cc:1034] running on GCE node
I20260811 22:13:11.665267 27258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:11.666630 27258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:11.667858 27258 hybrid_clock.cc:648] HybridClock initialized: now 1786486391667824 us; error 42 us; skew 500 ppm
I20260811 22:13:11.668195 27258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:13:11 dist-test-slave-68xk krb5kdc[27241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:11.671296 27258 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:13:11.672753 27258 webserver.cc:460] Webserver started at http://127.10.173.190:34951/ using document root <none> and password file <none>
I20260811 22:13:11.673429 27258 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:11.673573 27258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:11.673887 27258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:11.676880 27258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "347d393fb2d24dfe92e9ce8974b1bd0e"
format_stamp: "Formatted at 2026-08-11 22:13:11 on dist-test-slave-68xk"
server_key: "104a2aea6161ab0b2a950e84708ff52e"
server_key_iv: "4a7cbe01358ac2fcdaf560dbf8f042c9"
server_key_version: "encryptionkey@0"
I20260811 22:13:11.677650 27258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "347d393fb2d24dfe92e9ce8974b1bd0e"
format_stamp: "Formatted at 2026-08-11 22:13:11 on dist-test-slave-68xk"
server_key: "104a2aea6161ab0b2a950e84708ff52e"
server_key_iv: "4a7cbe01358ac2fcdaf560dbf8f042c9"
server_key_version: "encryptionkey@0"
I20260811 22:13:11.682878 27258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 22:13:11.686405 27287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:11.687783 27258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:13:11.687963 27258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "347d393fb2d24dfe92e9ce8974b1bd0e"
format_stamp: "Formatted at 2026-08-11 22:13:11 on dist-test-slave-68xk"
server_key: "104a2aea6161ab0b2a950e84708ff52e"
server_key_iv: "4a7cbe01358ac2fcdaf560dbf8f042c9"
server_key_version: "encryptionkey@0"
I20260811 22:13:11.688203 27258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:11.723426 27258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:11.727949 27258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:11.728272 27258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:11.748116 27258 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:46583
I20260811 22:13:11.748139 27379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:46583 every 8 connection(s)
I20260811 22:13:11.749609 27258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:11.753482 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 27258
I20260811 22:13:11.753623 27380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:11.753722 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:13:11.754675 10934 external_mini_cluster.cc:1428] Setting key 3a6000c04b4b812100bf24ae5aa5df04
I20260811 22:13:11.769462 27380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e: Bootstrap starting.
Aug 11 22:13:11 dist-test-slave-68xk krb5kdc[27241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:13:11.773337 27380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:11.774667 27380 log.cc:824] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:11.778026 27380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e: No bootstrap required, opened a new log
I20260811 22:13:11.781494 27384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:11.756857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35568 (local address 127.10.173.190:46583)
0811 22:13:11.757757 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:11.757773 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:11.757820 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:13:11.761533 (+ 3713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:11.761576 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:11.762048 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:11.762452 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:11.764331 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:11.765668 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:11.767584 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:11.768277 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:11.768417 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:11.773256 (+ 4839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:11.773264 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:11.773280 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:11.773327 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:11.777043 (+ 3716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:11.777823 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:11.777830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:11.777837 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:11.778022 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:11.778481 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:11.778487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:11.778495 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:11.779072 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:11.779454 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:11.779954 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 22:13:11.780393 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":459,"threads_started":1}
I20260811 22:13:11.786490 27380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347d393fb2d24dfe92e9ce8974b1bd0e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46583 } }
I20260811 22:13:11.786890 27380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:11.787000 27380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 347d393fb2d24dfe92e9ce8974b1bd0e, State: Initialized, Role: FOLLOWER
I20260811 22:13:11.787855 27380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347d393fb2d24dfe92e9ce8974b1bd0e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46583 } }
I20260811 22:13:11.788153 27380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:11.788275 27380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:11.788396 27380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:11.790676 27380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347d393fb2d24dfe92e9ce8974b1bd0e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46583 } }
I20260811 22:13:11.791183 27380 leader_election.cc:302] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 347d393fb2d24dfe92e9ce8974b1bd0e; no voters:
I20260811 22:13:11.791906 27380 leader_election.cc:288] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:11.792042 27389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:11.793257 27389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [term 1 LEADER]: Becoming Leader. State: Replica: 347d393fb2d24dfe92e9ce8974b1bd0e, State: Running, Role: LEADER
I20260811 22:13:11.793880 27389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347d393fb2d24dfe92e9ce8974b1bd0e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46583 } }
I20260811 22:13:11.794189 27380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:11.796765 27394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 347d393fb2d24dfe92e9ce8974b1bd0e. Latest consensus state: current_term: 1 leader_uuid: "347d393fb2d24dfe92e9ce8974b1bd0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347d393fb2d24dfe92e9ce8974b1bd0e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46583 } } }
I20260811 22:13:11.796828 27393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "347d393fb2d24dfe92e9ce8974b1bd0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "347d393fb2d24dfe92e9ce8974b1bd0e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46583 } } }
I20260811 22:13:11.797152 27394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:11.797257 27393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:11.799468 27399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:11.804773 27399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:11.814855 27399 catalog_manager.cc:1347] Generated new cluster ID: b9ff2ffdfbd947d4a94b014a08057a76
I20260811 22:13:11.814997 27399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:11.825839 27399 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:11.827420 27399 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:11.840305 27399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e: Generated new TSK 0
I20260811 22:13:11.841282 27399 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:13:11.915504 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46583
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:12.258991 27426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:12.259248 27426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:12.259408 27426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:12.266505 27426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:12.266626 27426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:12.266814 27426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:13:12.276504 27426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:12.277823 27426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:12.279220 27426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:12.292197 27439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:12.292222 27437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:12.292506 27443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:12.293107 27426 server_base.cc:1034] running on GCE node
I20260811 22:13:12.293682 27426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:12.294641 27426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:12.295902 27426 hybrid_clock.cc:648] HybridClock initialized: now 1786486392295853 us; error 43 us; skew 500 ppm
I20260811 22:13:12.296216 27426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:13:12 dist-test-slave-68xk krb5kdc[27241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486392, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:12.299422 27426 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:13:12.300836 27426 webserver.cc:460] Webserver started at http://127.10.173.129:41579/ using document root <none> and password file <none>
I20260811 22:13:12.301498 27426 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:12.301630 27426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:12.301934 27426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:12.304813 27426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "29ff71aa1e2146c496d9cc7786ecb744"
format_stamp: "Formatted at 2026-08-11 22:13:12 on dist-test-slave-68xk"
server_key: "dc54eacb904a49eab8fdd4bb42a9cbbd"
server_key_iv: "1a5bee4f1daba60193afcfdedea4b230"
server_key_version: "encryptionkey@0"
I20260811 22:13:12.305583 27426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "29ff71aa1e2146c496d9cc7786ecb744"
format_stamp: "Formatted at 2026-08-11 22:13:12 on dist-test-slave-68xk"
server_key: "dc54eacb904a49eab8fdd4bb42a9cbbd"
server_key_iv: "1a5bee4f1daba60193afcfdedea4b230"
server_key_version: "encryptionkey@0"
I20260811 22:13:12.310624 27426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 22:13:12.313843 27456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:12.315145 27426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:12.315325 27426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "29ff71aa1e2146c496d9cc7786ecb744"
format_stamp: "Formatted at 2026-08-11 22:13:12 on dist-test-slave-68xk"
server_key: "dc54eacb904a49eab8fdd4bb42a9cbbd"
server_key_iv: "1a5bee4f1daba60193afcfdedea4b230"
server_key_version: "encryptionkey@0"
I20260811 22:13:12.315510 27426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:12.345291 27426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:12.349581 27426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:12.349891 27426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:12.350931 27426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:12.352283 27426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:12.352382 27426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:12.352520 27426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:12.352598 27426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:12.387686 27426 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:34871
I20260811 22:13:12.387773 27655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:34871 every 8 connection(s)
I20260811 22:13:12.388995 27426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:12.396289 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 27426
I20260811 22:13:12.396517 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:13:12.397387 10934 external_mini_cluster.cc:1428] Setting key f67ec0e1ba6063c092d7fe916883e197
Aug 11 22:13:12 dist-test-slave-68xk krb5kdc[27241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486392, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:13:12.410297 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:12.391592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:42983 (local address 127.10.173.190:46583)
0811 22:13:12.392019 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:12.392023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:12.393071 (+ 1048us) server_negotiation.cc:407] Connection header received
0811 22:13:12.395102 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:12.395108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:12.395233 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:12.395358 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:12.397414 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:12.398174 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:12.400201 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:12.400656 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:12.400785 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:12.404358 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:12.404363 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:12.404366 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:12.404399 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:12.407182 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:12.407919 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:12.407929 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:12.407932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:12.408049 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:12.408496 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:12.408501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:12.408509 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:12.408783 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:12.408940 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:12.409693 (+ 753us) server_negotiation.cc:299] Negotiation successful
0811 22:13:12.409920 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":209,"threads_started":1}
I20260811 22:13:12.411428 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:12.392145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46583 (local address 127.10.173.129:42983)
0811 22:13:12.392926 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:12.392963 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:12.394809 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:12.395592 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:12.395601 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:12.396335 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:12.397202 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:12.397231 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:12.398342 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:12.398347 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:12.400041 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:12.400053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:12.400887 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:12.400895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:12.401147 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:12.401831 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:12.401855 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:12.404083 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:12.407335 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:12.407344 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:12.407362 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:12.407790 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:12.408169 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:12.408174 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:12.408178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:12.408376 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:12.408915 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:12.408922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:12.409281 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 22:13:12.409562 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 22:13:12.410111 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":280,"threads_started":1}
I20260811 22:13:12.413007 27656 heartbeater.cc:347] Connected to a master server at 127.10.173.190:46583
I20260811 22:13:12.413347 27656 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:12.414281 27656 heartbeater.cc:511] Master 127.10.173.190:46583 requested a full tablet report, sending...
I20260811 22:13:12.416564 27323 ts_manager.cc:194] Registered new tserver with Master: 29ff71aa1e2146c496d9cc7786ecb744 (127.10.173.129:34871)
I20260811 22:13:12.417995 27323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:42983
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:13:12.465585 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46583
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:12.811362 27663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:12.811645 27663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:12.811794 27663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:12.820226 27663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:12.820367 27663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:12.820570 27663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:13:12.831545 27663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:12.832986 27663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:12.834955 27663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:12.849066 27674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:12.849030 27676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:12.849110 27678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:12.850324 27663 server_base.cc:1034] running on GCE node
I20260811 22:13:12.851056 27663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:12.852035 27663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:12.853269 27663 hybrid_clock.cc:648] HybridClock initialized: now 1786486392853224 us; error 55 us; skew 500 ppm
I20260811 22:13:12.853600 27663 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:13:12 dist-test-slave-68xk krb5kdc[27241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486392, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:12.861063 27663 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:13:12.862691 27663 webserver.cc:460] Webserver started at http://127.10.173.130:32775/ using document root <none> and password file <none>
I20260811 22:13:12.863366 27663 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:12.863471 27663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:12.863744 27663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:12.866786 27663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6"
format_stamp: "Formatted at 2026-08-11 22:13:12 on dist-test-slave-68xk"
server_key: "7e61f47f71d17667cc9c8a173281e1ac"
server_key_iv: "6b4bfdb5038b31b49943acae612a135c"
server_key_version: "encryptionkey@0"
I20260811 22:13:12.867538 27663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6"
format_stamp: "Formatted at 2026-08-11 22:13:12 on dist-test-slave-68xk"
server_key: "7e61f47f71d17667cc9c8a173281e1ac"
server_key_iv: "6b4bfdb5038b31b49943acae612a135c"
server_key_version: "encryptionkey@0"
I20260811 22:13:12.872663 27663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:13:12.876161 27686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:12.877502 27663 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:13:12.877694 27663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6"
format_stamp: "Formatted at 2026-08-11 22:13:12 on dist-test-slave-68xk"
server_key: "7e61f47f71d17667cc9c8a173281e1ac"
server_key_iv: "6b4bfdb5038b31b49943acae612a135c"
server_key_version: "encryptionkey@0"
I20260811 22:13:12.877887 27663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:12.901041 27663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:12.905660 27663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:12.905980 27663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:12.907104 27663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:12.908478 27663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:12.908557 27663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:12.908704 27663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:12.908756 27663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:12.945544 27663 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:36631
I20260811 22:13:12.945636 27848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:36631 every 8 connection(s)
I20260811 22:13:12.946995 27663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:12.954537 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 27663
I20260811 22:13:12.954746 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:13:12.955367 10934 external_mini_cluster.cc:1428] Setting key 544bde555bfb5c4de6b6a03d18abcb86
Aug 11 22:13:12 dist-test-slave-68xk krb5kdc[27241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486392, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:13:12.969594 27851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:12.949769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:37219 (local address 127.10.173.190:46583)
0811 22:13:12.950292 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:12.950300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:12.951272 (+ 972us) server_negotiation.cc:407] Connection header received
0811 22:13:12.953069 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:12.953074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:12.953149 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:12.953256 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:12.954809 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:12.956004 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:12.957728 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:12.958494 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:12.958647 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:12.962016 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:12.962026 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:12.962034 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:12.962084 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:12.966003 (+ 3919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:12.966834 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:12.966843 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:12.966850 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:12.967032 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:12.967479 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:12.967487 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:12.967493 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:12.967925 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:12.968085 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:12.968759 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 22:13:12.969051 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":239,"threads_started":1}
I20260811 22:13:12.970160 27852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:12.950303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46583 (local address 127.10.173.130:37219)
0811 22:13:12.951104 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:12.951148 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:12.952812 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:12.953447 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:12.953456 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:12.953948 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:12.954587 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:12.954606 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:12.956168 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:12.956177 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:12.957587 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:12.957599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:12.958712 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:12.958717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:12.958916 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:12.959613 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:12.959635 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:12.961778 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:12.966173 (+ 4395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:12.966183 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:12.966195 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:12.966716 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:12.967165 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:12.967169 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:12.967171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:12.967356 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:12.968058 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:12.968063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:12.968375 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 22:13:12.968596 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 22:13:12.968993 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":288,"threads_started":1}
I20260811 22:13:12.971797 27849 heartbeater.cc:347] Connected to a master server at 127.10.173.190:46583
I20260811 22:13:12.972131 27849 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:12.972901 27849 heartbeater.cc:511] Master 127.10.173.190:46583 requested a full tablet report, sending...
I20260811 22:13:12.974498 27323 ts_manager.cc:194] Registered new tserver with Master: c8734bc7ae7f40ed9b5c9cae14c87bc6 (127.10.173.130:36631)
I20260811 22:13:12.975368 27323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:37219
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:13:13.026156 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46583
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:39735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:13.386590 27857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:13.386926 27857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:13.387087 27857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:13.394564 27857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:13.394716 27857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:13.394874 27857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:13:13.404587 27857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:39735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46583
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:13:13.405920 27857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:13.407275 27857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:13.420228 27866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:13.420281 27869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:13.420331 27865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:13.421212 27857 server_base.cc:1034] running on GCE node
I20260811 22:13:13.421830 27656 heartbeater.cc:503] Master 127.10.173.190:46583 was elected leader, sending a full tablet report...
I20260811 22:13:13.421950 27857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:13.423169 27857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:13.424445 27857 hybrid_clock.cc:648] HybridClock initialized: now 1786486393424365 us; error 86 us; skew 500 ppm
I20260811 22:13:13.424752 27857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:13:13 dist-test-slave-68xk krb5kdc[27241](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486393, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:13.428118 27857 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:13:13.429998 27857 webserver.cc:460] Webserver started at http://127.10.173.131:36417/ using document root <none> and password file <none>
I20260811 22:13:13.430812 27857 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:13.430962 27857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:13.431293 27857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:13.434478 27857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "17acf0950ef04a548457794ff6414649"
format_stamp: "Formatted at 2026-08-11 22:13:13 on dist-test-slave-68xk"
server_key: "49bed9c706d52ff746a646e54bc140a7"
server_key_iv: "3d6d5f6f07b534eead290876c53b54a3"
server_key_version: "encryptionkey@0"
I20260811 22:13:13.435303 27857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "17acf0950ef04a548457794ff6414649"
format_stamp: "Formatted at 2026-08-11 22:13:13 on dist-test-slave-68xk"
server_key: "49bed9c706d52ff746a646e54bc140a7"
server_key_iv: "3d6d5f6f07b534eead290876c53b54a3"
server_key_version: "encryptionkey@0"
I20260811 22:13:13.440701 27857 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 22:13:13.444660 27881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:13.446306 27857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:13.446548 27857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "17acf0950ef04a548457794ff6414649"
format_stamp: "Formatted at 2026-08-11 22:13:13 on dist-test-slave-68xk"
server_key: "49bed9c706d52ff746a646e54bc140a7"
server_key_iv: "3d6d5f6f07b534eead290876c53b54a3"
server_key_version: "encryptionkey@0"
I20260811 22:13:13.446790 27857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:13.468971 27857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:13.474056 27857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:13.474396 27857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:13.475520 27857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:13.477056 27857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:13.477169 27857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:13.477329 27857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:13.477406 27857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:13.515672 27857 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:38359
I20260811 22:13:13.515705 28057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:38359 every 8 connection(s)
I20260811 22:13:13.517110 27857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:13.525688 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 27857
I20260811 22:13:13.525904 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:13:13.526351 10934 external_mini_cluster.cc:1428] Setting key 6394f3ed2cff05dd6c8c6ccf61eb6a8d
Aug 11 22:13:13 dist-test-slave-68xk krb5kdc[27241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486393, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:13:13.537209 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.519956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:35389 (local address 127.10.173.190:46583)
0811 22:13:13.520478 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:13.520482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:13.521309 (+ 827us) server_negotiation.cc:407] Connection header received
0811 22:13:13.523137 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:13.523143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:13.523237 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:13.523355 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:13.524848 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.525670 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.527458 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.527949 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.528092 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.531757 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:13.531762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:13.531765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:13.531799 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:13.534312 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:13.535263 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:13.535267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:13.535270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:13.535388 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:13.535758 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:13.535761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:13.535763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:13.536009 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:13.536124 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:13.536793 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 22:13:13.536947 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":287,"threads_started":1}
I20260811 22:13:13.538148 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.520398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46583 (local address 127.10.173.131:35389)
0811 22:13:13.521125 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:13.521162 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:13.522880 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:13.523592 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:13.523600 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:13.524118 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:13.524670 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.524687 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.525834 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.525844 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.527319 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.527331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.528190 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.528195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.528511 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.529190 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:13.529242 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:13.531562 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:13.534488 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:13.534494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:13.534506 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:13.535126 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:13.535482 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:13.535485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:13.535487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:13.535660 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:13.536161 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:13.536167 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:13.536472 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 22:13:13.536702 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 22:13:13.537082 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":292,"threads_started":1}
I20260811 22:13:13.539618 28058 heartbeater.cc:347] Connected to a master server at 127.10.173.190:46583
I20260811 22:13:13.539947 28058 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:13.540853 28058 heartbeater.cc:511] Master 127.10.173.190:46583 requested a full tablet report, sending...
I20260811 22:13:13.542495 27323 ts_manager.cc:194] Registered new tserver with Master: 17acf0950ef04a548457794ff6414649 (127.10.173.131:38359)
I20260811 22:13:13.543337 27323 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:35389
I20260811 22:13:13.554163 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:13:13.573196 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.558784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35584 (local address 127.10.173.190:46583)
0811 22:13:13.558945 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:13.558949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:13.559479 (+ 530us) server_negotiation.cc:407] Connection header received
0811 22:13:13.559687 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:13.559696 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:13.559762 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:13.559865 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:13.561565 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.562394 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.563872 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.564338 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.564489 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.567168 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:13.567174 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:13.567178 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:13.567212 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:13.570102 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:13.570914 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:13.570921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:13.570925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:13.571060 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:13.571452 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:13.571458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:13.571462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:13.571851 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:13.572140 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:13.572632 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 22:13:13.572872 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:13:13.577472 27323 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:13:13.585017 27323 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:13:13.634582 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.598904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49208 (local address 127.10.173.129:34871)
0811 22:13:13.624986 (+ 26082us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:13.624994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:13.625040 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:13:13.625171 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:13.625178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:13.625464 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:13.625711 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:13.626979 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.628769 (+ 1790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.631683 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.633206 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.633419 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.633611 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:13.634010 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 22:13:13.634192 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25913,"thread_start_us":27252,"threads_started":1}
I20260811 22:13:13.634579 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.598349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:34871 (local address 127.0.0.1:49208)
0811 22:13:13.623258 (+ 24909us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:13.623318 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:13.623567 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:13.626025 (+ 2458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:13.626032 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:13.626155 (+ 123us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:13.626701 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.626723 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.628924 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.628928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.631499 (+ 2571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.631514 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.633426 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.633433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.633609 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.633671 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 22:13:13.633794 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 22:13:13.633995 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24528,"mutex_wait_us":24,"thread_start_us":28759,"threads_started":1}
I20260811 22:13:13.634711 28075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.598349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:38359 (local address 127.0.0.1:54006)
0811 22:13:13.626940 (+ 28591us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:13.626961 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:13.627078 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:13.627855 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:13.627860 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:13.627889 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:13.628319 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.628332 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.630181 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.630188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.631860 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.631874 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.633778 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.633784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.633960 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.633993 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:13:13.634109 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 22:13:13.634233 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28464,"mutex_wait_us":769,"thread_start_us":24667,"threads_started":1}
I20260811 22:13:13.635346 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.599019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45068 (local address 127.10.173.130:36631)
0811 22:13:13.624973 (+ 25954us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:13.624985 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:13.625943 (+ 958us) server_negotiation.cc:407] Connection header received
0811 22:13:13.626041 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:13.626048 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:13.626278 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:13.626496 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:13.627938 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.629624 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.632201 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.633605 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.633777 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.633932 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:13.634821 (+ 889us) server_negotiation.cc:299] Negotiation successful
0811 22:13:13.635002 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25779,"thread_start_us":25998,"threads_started":1}
I20260811 22:13:13.635461 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.598606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54006 (local address 127.10.173.131:38359)
0811 22:13:13.627254 (+ 28648us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:13.627291 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:13.627331 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:13:13.627425 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:13.627432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:13.627659 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:13.627846 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:13.628532 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.630024 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.632074 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.633507 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.633718 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.633880 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:13.634965 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0811 22:13:13.635130 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28456,"thread_start_us":28266,"threads_started":1}
I20260811 22:13:13.635653 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.598779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:36631 (local address 127.0.0.1:45068)
0811 22:13:13.625667 (+ 26888us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:13.625688 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:13.625810 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:13.627301 (+ 1491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:13.627309 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:13.627338 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:13.627730 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.627739 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.629770 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.629776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.632009 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.632024 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.634091 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.634097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.634287 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.634334 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 22:13:13.634457 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 22:13:13.634625 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26711,"mutex_wait_us":2,"thread_start_us":26575,"threads_started":1}
I20260811 22:13:13.638993 27536 tablet_service.cc:1467] Processing CreateTablet for tablet e782e68532f443b6a9b3463c1ec6243e (DEFAULT_TABLE table=test-table [id=68b2a5d3d71343ba95b872d1dbdfc667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:13.639135 27955 tablet_service.cc:1467] Processing CreateTablet for tablet e782e68532f443b6a9b3463c1ec6243e (DEFAULT_TABLE table=test-table [id=68b2a5d3d71343ba95b872d1dbdfc667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:13.639133 27759 tablet_service.cc:1467] Processing CreateTablet for tablet e782e68532f443b6a9b3463c1ec6243e (DEFAULT_TABLE table=test-table [id=68b2a5d3d71343ba95b872d1dbdfc667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:13.640282 27536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e782e68532f443b6a9b3463c1ec6243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:13.640468 27759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e782e68532f443b6a9b3463c1ec6243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:13.640468 27955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e782e68532f443b6a9b3463c1ec6243e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:13.653419 28085 tablet_bootstrap.cc:492] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: Bootstrap starting.
I20260811 22:13:13.656769 28085 tablet_bootstrap.cc:654] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:13.657816 28085 log.cc:824] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:13.660940 28086 tablet_bootstrap.cc:492] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: Bootstrap starting.
I20260811 22:13:13.661307 28087 tablet_bootstrap.cc:492] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: Bootstrap starting.
I20260811 22:13:13.664162 28087 tablet_bootstrap.cc:654] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:13.665084 28086 tablet_bootstrap.cc:654] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:13.665511 28087 log.cc:824] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:13.666234 28086 log.cc:824] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:13.667153 28085 tablet_bootstrap.cc:492] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: No bootstrap required, opened a new log
I20260811 22:13:13.667438 28085 ts_tablet_manager.cc:1397] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260811 22:13:13.668627 28087 tablet_bootstrap.cc:492] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: No bootstrap required, opened a new log
I20260811 22:13:13.668900 28087 ts_tablet_manager.cc:1397] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 22:13:13.672452 28086 tablet_bootstrap.cc:492] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: No bootstrap required, opened a new log
I20260811 22:13:13.672719 28086 ts_tablet_manager.cc:1397] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 22:13:13.678696 28085 raft_consensus.cc:348] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 } } peers { permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 } } peers { permanent_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36631 } }
I20260811 22:13:13.679231 28085 raft_consensus.cc:374] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:13.679365 28085 raft_consensus.cc:729] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29ff71aa1e2146c496d9cc7786ecb744, State: Initialized, Role: FOLLOWER
I20260811 22:13:13.679461 28087 raft_consensus.cc:348] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 } } peers { permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 } } peers { permanent_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36631 } }
I20260811 22:13:13.679920 28087 raft_consensus.cc:374] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:13.680042 28087 raft_consensus.cc:729] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8734bc7ae7f40ed9b5c9cae14c87bc6, State: Initialized, Role: FOLLOWER
I20260811 22:13:13.680099 28085 consensus_queue.cc:260] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 } } peers { permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 } } peers { permanent_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36631 } }
I20260811 22:13:13.680706 28087 consensus_queue.cc:260] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 } } peers { permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 } } peers { permanent_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36631 } }
I20260811 22:13:13.680609 28086 raft_consensus.cc:348] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 } } peers { permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 } } peers { permanent_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36631 } }
I20260811 22:13:13.681071 28086 raft_consensus.cc:374] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:13.681239 28086 raft_consensus.cc:729] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17acf0950ef04a548457794ff6414649, State: Initialized, Role: FOLLOWER
I20260811 22:13:13.682040 28086 consensus_queue.cc:260] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 } } peers { permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 } } peers { permanent_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36631 } }
I20260811 22:13:13.682629 28085 ts_tablet_manager.cc:1428] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260811 22:13:13.683894 28058 heartbeater.cc:503] Master 127.10.173.190:46583 was elected leader, sending a full tablet report...
I20260811 22:13:13.684474 28086 ts_tablet_manager.cc:1428] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 22:13:13.685259 28087 ts_tablet_manager.cc:1428] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: Time spent starting tablet: real 0.016s user 0.007s sys 0.007s
I20260811 22:13:13.685708 27849 heartbeater.cc:503] Master 127.10.173.190:46583 was elected leader, sending a full tablet report...
W20260811 22:13:13.699054 27850 tablet.cc:2369] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:13:13.741475 28097 raft_consensus.cc:482] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:13.741858 28097 raft_consensus.cc:504] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 } } peers { permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 } } peers { permanent_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36631 } }
I20260811 22:13:13.743862 28097 leader_election.cc:288] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17acf0950ef04a548457794ff6414649 (127.10.173.131:38359), 29ff71aa1e2146c496d9cc7786ecb744 (127.10.173.129:34871)
I20260811 22:13:13.753351 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.744209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:34871 (local address 127.10.173.130:54079)
0811 22:13:13.745274 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:13.745301 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:13.745494 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:13.745921 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:13.745925 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:13.745968 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:13.746590 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.746604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.748237 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.748242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.750254 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.750268 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.752336 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.752344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.752549 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.752599 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 22:13:13.752705 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:13:13.752835 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":142,"thread_start_us":641,"threads_started":1}
I20260811 22:13:13.753580 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.744740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:54079 (local address 127.10.173.129:34871)
0811 22:13:13.745047 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:13.745056 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:13.745430 (+ 374us) server_negotiation.cc:407] Connection header received
0811 22:13:13.745677 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:13.745683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:13.745777 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:13.745897 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:13.746740 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.748023 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.750441 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.751881 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.752057 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.752149 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:13.752773 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 22:13:13.753326 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 22:13:13.753997 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.744357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:41071 (local address 127.10.173.131:38359)
0811 22:13:13.744620 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:13.744630 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:13.745665 (+ 1035us) server_negotiation.cc:407] Connection header received
0811 22:13:13.745788 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:13.745793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:13.745887 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:13.745984 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:13.746697 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.748007 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.750761 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.752178 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.752351 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.752430 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:13.753568 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0811 22:13:13.753703 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:13:13.754781 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.744336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:38359 (local address 127.10.173.130:41071)
0811 22:13:13.745554 (+ 1218us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:13.745570 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:13.745679 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:13.746000 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:13.746006 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:13.746034 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:13.746488 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.746508 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.748161 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.748167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.750611 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:13.750627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.753118 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.753125 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:13.753332 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.753386 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 22:13:13.754284 (+ 898us) client_negotiation.cc:240] Negotiation successful
0811 22:13:13.754470 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"mutex_wait_us":69,"thread_start_us":872,"threads_started":1}
I20260811 22:13:13.754889 27573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e782e68532f443b6a9b3463c1ec6243e" candidate_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29ff71aa1e2146c496d9cc7786ecb744" is_pre_election: true
I20260811 22:13:13.755511 27573 raft_consensus.cc:2454] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8734bc7ae7f40ed9b5c9cae14c87bc6 in term 0.
I20260811 22:13:13.755896 27986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e782e68532f443b6a9b3463c1ec6243e" candidate_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17acf0950ef04a548457794ff6414649" is_pre_election: true
I20260811 22:13:13.756346 27687 leader_election.cc:302] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29ff71aa1e2146c496d9cc7786ecb744, c8734bc7ae7f40ed9b5c9cae14c87bc6; no voters:
I20260811 22:13:13.756498 27986 raft_consensus.cc:2454] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8734bc7ae7f40ed9b5c9cae14c87bc6 in term 0.
I20260811 22:13:13.756836 28097 raft_consensus.cc:2781] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:13:13.756997 28097 raft_consensus.cc:482] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:13:13.757117 28097 raft_consensus.cc:3037] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:13.760375 28097 raft_consensus.cc:504] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 } } peers { permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 } } peers { permanent_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36631 } }
I20260811 22:13:13.761081 28097 leader_election.cc:288] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [CANDIDATE]: Term 1 election: Requested vote from peers 17acf0950ef04a548457794ff6414649 (127.10.173.131:38359), 29ff71aa1e2146c496d9cc7786ecb744 (127.10.173.129:34871)
I20260811 22:13:13.761583 27573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e782e68532f443b6a9b3463c1ec6243e" candidate_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29ff71aa1e2146c496d9cc7786ecb744"
I20260811 22:13:13.761636 27986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e782e68532f443b6a9b3463c1ec6243e" candidate_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17acf0950ef04a548457794ff6414649"
I20260811 22:13:13.761781 27573 raft_consensus.cc:3037] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:13.761809 27986 raft_consensus.cc:3037] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:13.764272 27986 raft_consensus.cc:2454] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8734bc7ae7f40ed9b5c9cae14c87bc6 in term 1.
I20260811 22:13:13.764737 27691 leader_election.cc:302] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17acf0950ef04a548457794ff6414649, c8734bc7ae7f40ed9b5c9cae14c87bc6; no voters:
I20260811 22:13:13.764881 27573 raft_consensus.cc:2454] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8734bc7ae7f40ed9b5c9cae14c87bc6 in term 1.
I20260811 22:13:13.765115 28097 raft_consensus.cc:2781] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:13.766119 28097 raft_consensus.cc:686] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 1 LEADER]: Becoming Leader. State: Replica: c8734bc7ae7f40ed9b5c9cae14c87bc6, State: Running, Role: LEADER
I20260811 22:13:13.766762 28097 consensus_queue.cc:237] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 } } peers { permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 } } peers { permanent_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36631 } }
W20260811 22:13:13.768842 28059 tablet.cc:2369] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:13:13.772240 27320 catalog_manager.cc:5495] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8734bc7ae7f40ed9b5c9cae14c87bc6 (127.10.173.130). New cstate: current_term: 1 leader_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8734bc7ae7f40ed9b5c9cae14c87bc6" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36631 } health_report { overall_health: HEALTHY } } }
I20260811 22:13:13.795801 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.790110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.10.173.130:36631)
0811 22:13:13.790313 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:13.790322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:13.790406 (+ 84us) server_negotiation.cc:407] Connection header received
0811 22:13:13.790685 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:13.790689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:13.790782 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:13.790941 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:13:13.791349 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.792118 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.793487 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.793916 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.794049 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.794622 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:13:13.795159 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:13:13.795317 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:13.795403 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 22:13:13.795485 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:13:13.804132 27573 raft_consensus.cc:1261] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744 [term 1 FOLLOWER]: Refusing update from remote peer c8734bc7ae7f40ed9b5c9cae14c87bc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:13.804134 27986 raft_consensus.cc:1261] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649 [term 1 FOLLOWER]: Refusing update from remote peer c8734bc7ae7f40ed9b5c9cae14c87bc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:13.805119 28104 consensus_queue.cc:1035] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29ff71aa1e2146c496d9cc7786ecb744" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 34871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:13.805459 28097 consensus_queue.cc:1035] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17acf0950ef04a548457794ff6414649" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 38359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:13.817739 28113 mvcc.cc:205] Tried to move back new op lower bound from 7317448269011648512 to 7317448268873977856. Current Snapshot: MvccSnapshot[applied={T|T < 7317448269011648512}]
I20260811 22:13:13.821729 28117 mvcc.cc:205] Tried to move back new op lower bound from 7317448269011648512 to 7317448268873977856. Current Snapshot: MvccSnapshot[applied={T|T < 7317448269011648512}]
I20260811 22:13:13.831176 27323 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:13.831435 27323 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:13.834875 27323 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 347d393fb2d24dfe92e9ce8974b1bd0e: Sending DeleteTablet for 3 replicas of tablet e782e68532f443b6a9b3463c1ec6243e
I20260811 22:13:13.836151 27536 tablet_service.cc:1514] Processing DeleteTablet for tablet e782e68532f443b6a9b3463c1ec6243e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:13 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:49208
I20260811 22:13:13.836308 27955 tablet_service.cc:1514] Processing DeleteTablet for tablet e782e68532f443b6a9b3463c1ec6243e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:13 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:54006
I20260811 22:13:13.836259 27759 tablet_service.cc:1514] Processing DeleteTablet for tablet e782e68532f443b6a9b3463c1ec6243e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:13 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:45068
I20260811 22:13:13.840050 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 27426
I20260811 22:13:13.861857 28139 tablet_replica.cc:330] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: stopping tablet replica
I20260811 22:13:13.862737 28139 raft_consensus.cc:2229] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:13:13.863804 28139 raft_consensus.cc:2258] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:13.869725 28137 tablet_replica.cc:330] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: stopping tablet replica
I20260811 22:13:13.870175 28138 tablet_replica.cc:330] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: stopping tablet replica
I20260811 22:13:13.870373 28137 raft_consensus.cc:2229] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:13.870642 28138 raft_consensus.cc:2229] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:13.871119 28137 raft_consensus.cc:2258] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:13.871172 28138 raft_consensus.cc:2258] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:13.872812 28139 ts_tablet_manager.cc:1905] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:13.873195 28138 ts_tablet_manager.cc:1905] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:13.874686 28137 ts_tablet_manager.cc:1905] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:13.880438 28138 ts_tablet_manager.cc:1918] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:13.880580 28139 ts_tablet_manager.cc:1918] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:13.880628 28138 log.cc:1196] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/e782e68532f443b6a9b3463c1ec6243e
I20260811 22:13:13.880786 28139 log.cc:1196] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/e782e68532f443b6a9b3463c1ec6243e
I20260811 22:13:13.881294 28138 ts_tablet_manager.cc:1939] T e782e68532f443b6a9b3463c1ec6243e P 29ff71aa1e2146c496d9cc7786ecb744: Deleting consensus metadata
I20260811 22:13:13.881460 28139 ts_tablet_manager.cc:1939] T e782e68532f443b6a9b3463c1ec6243e P c8734bc7ae7f40ed9b5c9cae14c87bc6: Deleting consensus metadata
I20260811 22:13:13.883540 27288 catalog_manager.cc:4841] TS 29ff71aa1e2146c496d9cc7786ecb744 (127.10.173.129:34871): tablet e782e68532f443b6a9b3463c1ec6243e (table test-table [id=68b2a5d3d71343ba95b872d1dbdfc667]) successfully deleted
I20260811 22:13:13.883571 27289 catalog_manager.cc:4841] TS c8734bc7ae7f40ed9b5c9cae14c87bc6 (127.10.173.130:36631): tablet e782e68532f443b6a9b3463c1ec6243e (table test-table [id=68b2a5d3d71343ba95b872d1dbdfc667]) successfully deleted
Aug 11 22:13:13 dist-test-slave-68xk krb5kdc[27241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:13:13.885900 28137 ts_tablet_manager.cc:1918] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:13.886091 28137 log.cc:1196] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/e782e68532f443b6a9b3463c1ec6243e
I20260811 22:13:13.886716 28137 ts_tablet_manager.cc:1939] T e782e68532f443b6a9b3463c1ec6243e P 17acf0950ef04a548457794ff6414649: Deleting consensus metadata
I20260811 22:13:13.888698 27290 catalog_manager.cc:4841] TS 17acf0950ef04a548457794ff6414649 (127.10.173.131:38359): tablet e782e68532f443b6a9b3463c1ec6243e (table test-table [id=68b2a5d3d71343ba95b872d1dbdfc667]) successfully deleted
I20260811 22:13:13.892351 28081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:13.841369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49220 (local address 127.10.173.129:34871)
0811 22:13:13.841715 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:13.841723 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:13.857763 (+ 16040us) server_negotiation.cc:407] Connection header received
0811 22:13:13.858484 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:13.858493 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:13.858628 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:13.858730 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:13.864157 (+ 5427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.865811 (+ 1654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.871653 (+ 5842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:13.872401 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:13.872510 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:13.885784 (+ 13274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:13.885798 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:13.885814 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:13.885853 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:13.889265 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:13.890075 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:13.890083 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:13.890086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:13.890270 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:13.890711 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:13.890721 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:13.890723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:13.891111 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:13.891559 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:13.891725 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:13:13.891972 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260811 22:13:13.892990 27649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:14.016125 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 27426
I20260811 22:13:14.049052 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 27663
Aug 11 22:13:14 dist-test-slave-68xk krb5kdc[27241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:13:14.066658 28082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:14.049900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45086 (local address 127.10.173.130:36631)
0811 22:13:14.050097 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:14.050103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:14.050135 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:13:14.050345 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:14.050349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:14.050479 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:14.050570 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:14.052285 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:14.053095 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:14.054970 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:14.055421 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:14.055518 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:14.060530 (+ 5012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:14.060544 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:14.060560 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:14.060599 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:14.063852 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:14.064544 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:14.064552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:14.064554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:14.064697 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:14.065083 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:14.065089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:14.065092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:14.065453 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:14.065830 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:14.066047 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 22:13:14.066269 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:13:14.067386 27843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:14.181645 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 27663
I20260811 22:13:14.207890 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 27857
Aug 11 22:13:14 dist-test-slave-68xk krb5kdc[27241](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:13:14.224454 28076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:14.208791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.10.173.131:38359)
0811 22:13:14.209039 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:14.209048 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:14.209073 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:13:14.209240 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:14.209246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:14.209348 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:14.209477 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:14.211162 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:14.211977 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:14.213227 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:14.213653 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:14.213762 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:14.218574 (+ 4812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:14.218598 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:14.218622 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:14.218696 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:14.221623 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:14.222491 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:14.222499 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:14.222502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:14.222658 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:14.223031 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:14.223034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:14.223036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:14.223415 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:14.223644 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:14.223960 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 22:13:14.224149 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 22:13:14.225292 28050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:14.318152 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 27857
I20260811 22:13:14.344463 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 27258
I20260811 22:13:14.345760 27372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:13:14.456354 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 27258
2026-08-11T22:13:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4904 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:14 dist-test-slave-68xk krb5kdc[28149](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:14 dist-test-slave-68xk krb5kdc[28149](info): set up 2 sockets
Aug 11 22:13:14 dist-test-slave-68xk krb5kdc[28149](info): commencing operation
krb5kdc: starting...
W20260811 22:13:15.919548 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.411s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:13:16 dist-test-slave-68xk krb5kdc[28149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:16Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.173.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.173.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/rangeradmin_127.10.173.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/rangeradmin_127.10.173.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.173.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.173.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/rangerlookup_127.10.173.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/rangerlookup_127.10.173.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.173.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.173.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/HTTP_127.10.173.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/HTTP_127.10.173.148@KRBTEST.COM.keytab.
I20260811 22:13:16.331233 10934 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskZXazun/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/postgres -l logfile start
2026-08-11 22:13:19.175 UTC [28191] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:13:19.176 UTC [28191] LOG: listening on IPv4 address "127.10.173.148", port 46811
2026-08-11 22:13:19.185 UTC [28191] LOG: listening on Unix socket "/tmp/.s.PGSQL.46811"
2026-08-11 22:13:19.210 UTC [28196] LOG: database system was shut down at 2026-08-11 22:13:17 UTC
2026-08-11 22:13:19.215 UTC [28191] LOG: database system is ready to accept connections
I20260811 22:13:20.441910 10934 mini_postgres.cc:96] Postgres bound to 46811
2026-08-11 22:13:20.468 UTC [28207] FATAL: database "slave" does not exist
127.10.173.148:46811 - accepting connections
I20260811 22:13:20.469077 10934 mini_ranger.cc:161] Starting Ranger...
I20260811 22:13:20.503384 10934 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:13:21.073091 10934 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:13:21.073282 10934 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-admin
2026-08-11 22:13:21,471 [I] DB FLAVOR :POSTGRES
2026-08-11 22:13:21,472 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:13:21,472 [I] Checking connection..
2026-08-11 22:13:21,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:13:21,762 [I] Checking connection passed.
2026-08-11 22:13:21,762 [I] --------- Verifying version history table ---------
2026-08-11 22:13:21,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:13:22,023 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:13:22,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:13:22,270 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:13:22,270 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:13:22,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:13:22.533 UTC [28312] WARNING: there is no transaction in progress
2026-08-11 22:13:22,542 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:13:22,542 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:13:22,805 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:13:22,805 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:13:22,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:13:23,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:13:23,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-68xk.c.gcp-upstream.internal','N') ;"
2026-08-11 22:13:23,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:13:23,821 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:13:23,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:13:24,071 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:13:24,072 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:13:24,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:13:24,569 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:13:24,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:13:25.584 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.601 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.619 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.636 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.653 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.676 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.736 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.743 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.759 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.769 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.779 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.794 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.804 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.811 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.819 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:25.826 UTC [28490] WARNING: there is no transaction in progress
2026-08-11 22:13:26,558 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:13:26,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:13:26,804 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:13:26,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:13:27,050 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:13:27,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:13:27,313 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:13:27,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:13:27,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:13:27,855 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-68xk.c.gcp-upstream.internal';"
2026-08-11 22:13:28,104 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:13:28,105 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:13:28,369 [I] DB_PATCHES have already been applied
I20260811 22:13:28.376135 10934 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-admin:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskZXazun/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:13:28.376286 10934 mini_ranger.cc:193] Using host: 127.10.173.148
I20260811 22:13:28.398690 10934 mini_ranger.cc:227] Ranger admin URL: http://127.10.173.148:54953
Aug 11, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:13:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:13:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:13:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:13:29.844486 10934 mini_ranger.cc:160] Time spent starting Ranger: real 9.375s user 0.001s sys 0.068s
Aug 11, 2026 10:13:50 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:13:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:13:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:13:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:13:51 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:13:52.318599 10934 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.173.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.173.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/rangerkms_127.10.173.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/rangerkms_127.10.173.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.173.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.173.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.173.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/HTTP_127.10.173.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.173.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/HTTP_127.10.173.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:13:52 dist-test-slave-68xk krb5kdc[28149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486432, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:13:52.538671 10934 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:13:52.568624 10934 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:13:53.573343 10934 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:13:53.573510 10934 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-kms
I20260811 22:13:53.573679 10934 mini_ranger_kms.cc:227] Using postgres at 127.10.173.148:46811
2026-08-11 22:13:54,087 [I] DB FLAVOR :POSTGRES
2026-08-11 22:13:54,087 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:13:54,087 [I] Checking connection
2026-08-11 22:13:54,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:13:54,338 [I] connection success
2026-08-11 22:13:54,338 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:13:54,338 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:13:54,338 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:13:54,573 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:13:54,573 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:13:54,573 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:13:54,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:46811/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:13:54,844 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:13:55.370376 10934 mini_ranger_kms.cc:310] Created kms service
I20260811 22:13:55.569294 10934 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:13:55.615483 10934 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:13:55 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486432, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:13:56.024205 10934 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:13:56.025774 10934 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-kms:/tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:13:56.025870 10934 mini_ranger_kms.cc:240] Using host: 127.10.173.148
I20260811 22:13:56.057963 10934 mini_ranger_kms.cc:276] Ranger KMS PID: 28800
I20260811 22:13:56.058197 10934 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.173.148:33843
Aug 11, 2026 10:13:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:13:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:13:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:13:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:13:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:13:57 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 22:13:58 dist-test-slave-68xk krb5kdc[28149](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486438, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.173.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:58.957376 10934 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.419s user 0.111s sys 0.445s
I20260811 22:13:58.957779 10934 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:13:58.958014 10934 mini_ranger_kms.cc:401] 127.10.173.148:33843/kms/v1/keys
Aug 11 22:13:59 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486438, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.173.148@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
Aug 11 22:13:59 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486438, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.173.148@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
Aug 11 22:14:02 dist-test-slave-68xk krb5kdc[28149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:14:02.098992 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:32809
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:44959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:32809
--ranger_config_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:33843/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:02.574625 28880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:02.575033 28880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:02.575201 28880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:02.582757 28880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:02.582866 28880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:02.583060 28880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:02.583165 28880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:02.583261 28880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:02.594313 28880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:44959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.173.148:33843/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:32809
--ranger_config_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:32809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:14:02.595784 28880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:02.597182 28880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:02.611953 28895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:02.612222 28891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:02.612195 28892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:02.613364 28880 server_base.cc:1034] running on GCE node
I20260811 22:14:02.614220 28880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:02.615818 28880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:02.617066 28880 hybrid_clock.cc:648] HybridClock initialized: now 1786486442617040 us; error 42 us; skew 500 ppm
I20260811 22:14:02.617444 28880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:14:02 dist-test-slave-68xk krb5kdc[28149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486442, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:02.620963 28880 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:14:02.622717 28880 webserver.cc:460] Webserver started at http://127.10.173.190:43741/ using document root <none> and password file <none>
I20260811 22:14:02.623410 28880 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:02.623559 28880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:02.623888 28880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:14:02 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486442, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:02.691052 28880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "355a35f82f4445e5af0f5142a0129cc4"
format_stamp: "Formatted at 2026-08-11 22:14:02 on dist-test-slave-68xk"
server_key: "d9f3f2c2b0e40bbd71d75c2fbc092c02"
server_key_iv: "ead7f977357a5c9d31d2d66583d17933"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:02.692232 28880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "355a35f82f4445e5af0f5142a0129cc4"
format_stamp: "Formatted at 2026-08-11 22:14:02 on dist-test-slave-68xk"
server_key: "d9f3f2c2b0e40bbd71d75c2fbc092c02"
server_key_iv: "ead7f977357a5c9d31d2d66583d17933"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:02.700213 28880 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260811 22:14:02.721453 28903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:02.723345 28880 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:14:02.723551 28880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "355a35f82f4445e5af0f5142a0129cc4"
format_stamp: "Formatted at 2026-08-11 22:14:02 on dist-test-slave-68xk"
server_key: "d9f3f2c2b0e40bbd71d75c2fbc092c02"
server_key_iv: "ead7f977357a5c9d31d2d66583d17933"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:02.723830 28880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:02.749225 28880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:02.753908 28880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:02.754212 28880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:02.778502 28880 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:32809
I20260811 22:14:02.778434 28987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:32809 every 8 connection(s)
I20260811 22:14:02.780031 28880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:02.784178 28988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:02.789198 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 28880
I20260811 22:14:02.789453 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:14:02.790242 10934 external_mini_cluster.cc:1428] Setting key f3d9d8e89ace21975bfd760596230628
I20260811 22:14:02.796751 28988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4: Bootstrap starting.
I20260811 22:14:02.800951 28988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:02.802078 28988 log.cc:824] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:02.805595 28988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4: No bootstrap required, opened a new log
Aug 11 22:14:02 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:14:02.815315 28988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355a35f82f4445e5af0f5142a0129cc4" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32809 } }
I20260811 22:14:02.815825 28988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:02.815825 28992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:02.792509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44408 (local address 127.10.173.190:32809)
0811 22:14:02.793624 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:02.793640 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:02.793700 (+ 60us) server_negotiation.cc:407] Connection header received
0811 22:14:02.796190 (+ 2490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:02.796223 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:02.796885 (+ 662us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:02.797353 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:02.798885 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:02.800957 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:02.802498 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:02.803512 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:02.803755 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:02.807900 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:02.807908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:02.807922 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:02.807963 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:02.811436 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:02.812203 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:02.812216 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:02.812221 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:02.812376 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:02.812793 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:02.812802 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:02.812804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:02.813364 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:02.813811 (+ 447us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:02.814223 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 22:14:02.814683 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":848,"thread_start_us":586,"threads_started":1}
I20260811 22:14:02.815984 28988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 355a35f82f4445e5af0f5142a0129cc4, State: Initialized, Role: FOLLOWER
I20260811 22:14:02.816809 28988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355a35f82f4445e5af0f5142a0129cc4" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32809 } }
I20260811 22:14:02.817137 28988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:02.817391 28988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:02.817682 28988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:02.820593 28988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355a35f82f4445e5af0f5142a0129cc4" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32809 } }
I20260811 22:14:02.821159 28988 leader_election.cc:302] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 355a35f82f4445e5af0f5142a0129cc4; no voters:
I20260811 22:14:02.821828 28988 leader_election.cc:288] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:02.821967 28994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:02.823374 28994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [term 1 LEADER]: Becoming Leader. State: Replica: 355a35f82f4445e5af0f5142a0129cc4, State: Running, Role: LEADER
I20260811 22:14:02.824303 28994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355a35f82f4445e5af0f5142a0129cc4" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32809 } }
I20260811 22:14:02.824483 28988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:02.827816 28999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 355a35f82f4445e5af0f5142a0129cc4. Latest consensus state: current_term: 1 leader_uuid: "355a35f82f4445e5af0f5142a0129cc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355a35f82f4445e5af0f5142a0129cc4" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32809 } } }
I20260811 22:14:02.828226 28999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:02.829021 28995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "355a35f82f4445e5af0f5142a0129cc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "355a35f82f4445e5af0f5142a0129cc4" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 32809 } } }
I20260811 22:14:02.829437 28995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:02.831209 29005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:02.838212 29005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:02.840900 28988 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:14:02.860685 29005 catalog_manager.cc:1347] Generated new cluster ID: f79b5e48a9f74ee186f9c2542d7a92c1
I20260811 22:14:02.860889 29005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:02.891844 29005 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:02.894447 29005 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:02.919320 29005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4: Generated new TSK 0
I20260811 22:14:02.920506 29005 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:14:03.787145 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:33843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32809
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:44959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:14:04 dist-test-slave-68xk krb5kdc[28149](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486444, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:14:04.386992 29064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:04.387328 29064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:04.387511 29064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:04.398051 29064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:04.398373 29064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:04.398703 29064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:14:04.414793 29064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:44959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.173.148:33843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:14:04.416438 29064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:04.418568 29064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:04.450855 29079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:04.457476 29080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:04.466388 29064 server_base.cc:1034] running on GCE node
W20260811 22:14:04.467311 29082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:04.468295 29064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:04.470836 29064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:04.472272 29064 hybrid_clock.cc:648] HybridClock initialized: now 1786486444472060 us; error 126 us; skew 500 ppm
I20260811 22:14:04.472785 29064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:14:04 dist-test-slave-68xk krb5kdc[28149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486444, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:04.479094 29064 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:14:04.489550 29064 webserver.cc:460] Webserver started at http://127.10.173.129:33831/ using document root <none> and password file <none>
I20260811 22:14:04.491039 29064 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:04.491331 29064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:04.491900 29064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:14:04 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486444, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:04.518107 29064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "5e6191162e4942dcb789f5e036352fa8"
format_stamp: "Formatted at 2026-08-11 22:14:04 on dist-test-slave-68xk"
server_key: "62725b8cf6ea5e53209b5fbb151a730a"
server_key_iv: "8e090c6fc5003920ee00bc7acb477d6d"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:04.520121 29064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "5e6191162e4942dcb789f5e036352fa8"
format_stamp: "Formatted at 2026-08-11 22:14:04 on dist-test-slave-68xk"
server_key: "62725b8cf6ea5e53209b5fbb151a730a"
server_key_iv: "8e090c6fc5003920ee00bc7acb477d6d"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:04.532191 29064 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.006s sys 0.006s
I20260811 22:14:04.556907 29089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:04.559003 29064 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260811 22:14:04.559464 29064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "5e6191162e4942dcb789f5e036352fa8"
format_stamp: "Formatted at 2026-08-11 22:14:04 on dist-test-slave-68xk"
server_key: "62725b8cf6ea5e53209b5fbb151a730a"
server_key_iv: "8e090c6fc5003920ee00bc7acb477d6d"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:04.559975 29064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:04.616451 29064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:04.623565 29064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:04.624177 29064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:04.630012 29064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:04.634573 29064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:04.634745 29064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 22:14:04.634994 29064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:04.635164 29064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:14:04 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486444, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:04.904855 29064 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:43887
I20260811 22:14:04.904910 29275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:43887 every 8 connection(s)
I20260811 22:14:04.907528 29064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:04.907974 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 29064
I20260811 22:14:04.908221 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:04.909250 10934 external_mini_cluster.cc:1428] Setting key 485871a6dcc074790ab175913f305920
Aug 11 22:14:04 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486444, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
I20260811 22:14:04.953691 29279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:04.919425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:48851 (local address 127.10.173.190:32809)
0811 22:14:04.931031 (+ 11606us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:04.931040 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:04.931070 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:14:04.932269 (+ 1199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:04.932277 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:04.932420 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:04.932517 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:04.935325 (+ 2808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:04.936704 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:04.939809 (+ 3105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:04.940699 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:04.940821 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:04.947049 (+ 6228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:04.947059 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:04.947067 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:04.947127 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:04.950010 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:04.950911 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:04.950919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:04.950927 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:04.951062 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:04.951634 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:04.951646 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:04.951649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:04.951930 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:04.952105 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:04.953021 (+ 916us) server_negotiation.cc:299] Negotiation successful
0811 22:14:04.953282 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11421,"thread_start_us":11265,"threads_started":1}
I20260811 22:14:04.954921 29278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:04.918380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32809 (local address 127.10.173.129:48851)
0811 22:14:04.927414 (+ 9034us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:04.927477 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:04.931836 (+ 4359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:04.933186 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:04.933201 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:04.934021 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:04.935081 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:04.935112 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:04.936968 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:04.936976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:04.939601 (+ 2625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:04.939620 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:04.941292 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:04.941303 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:04.941759 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:04.942756 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:04.942791 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:04.946677 (+ 3886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:04.950188 (+ 3511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:04.950198 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:04.950218 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:04.950766 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:04.951193 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:04.951198 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:04.951201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:04.951491 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:04.952072 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:04.952081 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:04.952556 (+ 475us) client_negotiation.cc:769] Sending connection context
0811 22:14:04.952899 (+ 343us) client_negotiation.cc:240] Negotiation successful
0811 22:14:04.953501 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8616,"thread_start_us":8290,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
I20260811 22:14:04.957113 29276 heartbeater.cc:347] Connected to a master server at 127.10.173.190:32809
I20260811 22:14:04.957482 29276 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:04.958616 29276 heartbeater.cc:511] Master 127.10.173.190:32809 requested a full tablet report, sending...
I20260811 22:14:04.961558 28926 ts_manager.cc:194] Registered new tserver with Master: 5e6191162e4942dcb789f5e036352fa8 (127.10.173.129:43887)
I20260811 22:14:04.963994 28926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:48851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:14:05.002948 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:33843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32809
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:44959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:14:05 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486444, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
W20260811 22:14:05.765832 29282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:05.766264 29282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:05.766580 29282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:05.778815 29282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:05.779109 29282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:05.779412 29282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:14:05.798566 29282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:44959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.173.148:33843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:14:05.800303 29282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:05.801858 29282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:05.820779 29307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:05.821552 29313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:05.821913 29309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:05.822805 29282 server_base.cc:1034] running on GCE node
I20260811 22:14:05.823452 29282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:05.824431 29282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:05.825670 29282 hybrid_clock.cc:648] HybridClock initialized: now 1786486445825614 us; error 78 us; skew 500 ppm
I20260811 22:14:05.826133 29282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:14:05 dist-test-slave-68xk krb5kdc[28149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486445, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:05.838083 29282 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:14:05.840763 29282 webserver.cc:460] Webserver started at http://127.10.173.130:35417/ using document root <none> and password file <none>
I20260811 22:14:05.841683 29282 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:05.841953 29282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:05.842525 29282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:14:05 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486445, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:05.863387 29282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "50f46af4c37445328ff9a155360e0ca2"
format_stamp: "Formatted at 2026-08-11 22:14:05 on dist-test-slave-68xk"
server_key: "4c5ced2b58b34debf04907b1ee01c15e"
server_key_iv: "800d69daf5673d5eab05b90e68c221bd"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:05.865078 29282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "50f46af4c37445328ff9a155360e0ca2"
format_stamp: "Formatted at 2026-08-11 22:14:05 on dist-test-slave-68xk"
server_key: "4c5ced2b58b34debf04907b1ee01c15e"
server_key_iv: "800d69daf5673d5eab05b90e68c221bd"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:05.875646 29282 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.004s
I20260811 22:14:05.897931 29323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:05.899700 29282 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 22:14:05.899909 29282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "50f46af4c37445328ff9a155360e0ca2"
format_stamp: "Formatted at 2026-08-11 22:14:05 on dist-test-slave-68xk"
server_key: "4c5ced2b58b34debf04907b1ee01c15e"
server_key_iv: "800d69daf5673d5eab05b90e68c221bd"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:05.900149 29282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:05.924710 29282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:05.931365 29282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:05.931938 29282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:05.934181 29282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:05.936918 29282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:05.937103 29282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:05.937382 29282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:05.937513 29282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:05.968591 29276 heartbeater.cc:503] Master 127.10.173.190:32809 was elected leader, sending a full tablet report...
I20260811 22:14:06.060556 29282 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:38193
I20260811 22:14:06.060817 29503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:38193 every 8 connection(s)
I20260811 22:14:06.062199 29282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:06.066272 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 29282
I20260811 22:14:06.066509 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:06.067055 10934 external_mini_cluster.cc:1428] Setting key 6676c701729967c1da632d9bc42beb74
Aug 11 22:14:06 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486445, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:14:06.095238 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:06.068495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:48197 (local address 127.10.173.190:32809)
0811 22:14:06.071334 (+ 2839us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:06.071351 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:06.071384 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:14:06.072917 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:06.072923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:06.073088 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:06.073219 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:06.075592 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:06.076590 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:06.078725 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:06.079340 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:06.079486 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:06.084506 (+ 5020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:06.084517 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:06.084523 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:06.084623 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:06.090012 (+ 5389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:06.091284 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:06.091292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:06.091299 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:06.091523 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:06.092235 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:06.092243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:06.092248 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:06.092787 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:06.093003 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:06.094192 (+ 1189us) server_negotiation.cc:299] Negotiation successful
0811 22:14:06.094504 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2639,"thread_start_us":2428,"threads_started":1}
I20260811 22:14:06.096688 29511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:06.067759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32809 (local address 127.10.173.130:48197)
0811 22:14:06.069465 (+ 1706us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:06.069506 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:06.072630 (+ 3124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:06.073571 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:06.073581 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:06.074332 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:06.075407 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:06.075428 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:06.076746 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:06.076754 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:06.078563 (+ 1809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:06.078576 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:06.079690 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:06.079702 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:06.080006 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:06.080918 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:06.080941 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:06.084161 (+ 3220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:06.090210 (+ 6049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:06.090221 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:06.090244 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:06.091110 (+ 866us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:06.091658 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:06.091664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:06.091668 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:06.092077 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:06.092945 (+ 868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:06.092954 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:06.093578 (+ 624us) client_negotiation.cc:769] Sending connection context
0811 22:14:06.093967 (+ 389us) client_negotiation.cc:240] Negotiation successful
0811 22:14:06.094822 (+ 855us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1353,"thread_start_us":896,"threads_started":1}
I20260811 22:14:06.099253 29509 heartbeater.cc:347] Connected to a master server at 127.10.173.190:32809
I20260811 22:14:06.099620 29509 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:06.100584 29509 heartbeater.cc:511] Master 127.10.173.190:32809 requested a full tablet report, sending...
I20260811 22:14:06.102720 28926 ts_manager.cc:194] Registered new tserver with Master: 50f46af4c37445328ff9a155360e0ca2 (127.10.173.130:38193)
I20260811 22:14:06.104096 28926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:48197
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:14:06.150369 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:33843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32809
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:44959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:06.803457 29518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:06.803963 29518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:06.804268 29518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:06.815871 29518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:06.816152 29518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:06.816494 29518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:14:06.830745 29518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:44959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.173.148:33843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:32809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:14:06.832333 29518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:06.833868 29518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:06.856282 29523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:06.857080 29537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:06.857326 29524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:06.858896 29518 server_base.cc:1034] running on GCE node
I20260811 22:14:06.859671 29518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:06.860865 29518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:06.862218 29518 hybrid_clock.cc:648] HybridClock initialized: now 1786486446862053 us; error 79 us; skew 500 ppm
I20260811 22:14:06.862809 29518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:14:06 dist-test-slave-68xk krb5kdc[28149](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486446, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:06.868225 29518 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:14:06.870587 29518 webserver.cc:460] Webserver started at http://127.10.173.131:38475/ using document root <none> and password file <none>
I20260811 22:14:06.871485 29518 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:06.871660 29518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:06.872032 29518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:14:06 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486446, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:06.888767 29518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "07c96d0135804a59838c6c8a97bdbf75"
format_stamp: "Formatted at 2026-08-11 22:14:06 on dist-test-slave-68xk"
server_key: "2296a0d52c466438003c55b4a5a82410"
server_key_iv: "46283eebe05eeeb2621c2e1e82d3dcb6"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:06.889967 29518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "07c96d0135804a59838c6c8a97bdbf75"
format_stamp: "Formatted at 2026-08-11 22:14:06 on dist-test-slave-68xk"
server_key: "2296a0d52c466438003c55b4a5a82410"
server_key_iv: "46283eebe05eeeb2621c2e1e82d3dcb6"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:06.898214 29518 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.005s
I20260811 22:14:06.917850 29549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:06.919720 29518 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:06.919978 29518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "07c96d0135804a59838c6c8a97bdbf75"
format_stamp: "Formatted at 2026-08-11 22:14:06 on dist-test-slave-68xk"
server_key: "2296a0d52c466438003c55b4a5a82410"
server_key_iv: "46283eebe05eeeb2621c2e1e82d3dcb6"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:06.920337 29518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:06.956249 29518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:06.962301 29518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:06.962874 29518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:06.964956 29518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:06.968469 29518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:06.968623 29518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:06.968883 29518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:06.969026 29518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:07.055749 29518 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:40703
I20260811 22:14:07.055894 29744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:40703 every 8 connection(s)
I20260811 22:14:07.057515 29518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:07.063234 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 29518
I20260811 22:14:07.063423 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:07.064317 10934 external_mini_cluster.cc:1428] Setting key 08bc8aff066c4e122a167f9e8f820e3a
Aug 11 22:14:07 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486446, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:14:07.088985 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.062297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:40055 (local address 127.10.173.190:32809)
0811 22:14:07.063181 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.063199 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.065664 (+ 2465us) server_negotiation.cc:407] Connection header received
0811 22:14:07.069183 (+ 3519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.069187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.069265 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.069357 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:07.071460 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.072370 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.074649 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.075244 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.075353 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.081334 (+ 5981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:07.081339 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:07.081343 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:07.081375 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:07.085322 (+ 3947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.086255 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.086260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.086262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.086540 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.087103 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.087107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.087109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.087479 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:07.087597 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.088361 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.088534 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":499,"threads_started":1}
I20260811 22:14:07.090246 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.063363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:32809 (local address 127.10.173.131:40055)
0811 22:14:07.065126 (+ 1763us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.065192 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:07.068836 (+ 3644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:07.069604 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:07.069612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:07.070362 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:07.071269 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.071289 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.072588 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.072592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.074499 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.074512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.075606 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.075613 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.075935 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.076993 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:07.077016 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:07.081040 (+ 4024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:07.085530 (+ 4490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:07.085539 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:07.085552 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:07.086108 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:07.086653 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:07.086657 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:07.086659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:07.086961 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:07.087593 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:07.087599 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:07.088019 (+ 420us) client_negotiation.cc:769] Sending connection context
0811 22:14:07.088263 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 22:14:07.088753 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1357,"thread_start_us":772,"threads_started":1}
I20260811 22:14:07.092024 29745 heartbeater.cc:347] Connected to a master server at 127.10.173.190:32809
I20260811 22:14:07.092427 29745 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:07.093477 29745 heartbeater.cc:511] Master 127.10.173.190:32809 requested a full tablet report, sending...
I20260811 22:14:07.095445 28925 ts_manager.cc:194] Registered new tserver with Master: 07c96d0135804a59838c6c8a97bdbf75 (127.10.173.131:40703)
I20260811 22:14:07.096647 28925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:40055
I20260811 22:14:07.105962 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:14:07.108021 29509 heartbeater.cc:503] Master 127.10.173.190:32809 was elected leader, sending a full tablet report...
I20260811 22:14:07.131201 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.113106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.10.173.190:32809)
0811 22:14:07.113318 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.113323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.114237 (+ 914us) server_negotiation.cc:407] Connection header received
0811 22:14:07.114483 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.114487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.114566 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.114680 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:07.116838 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.117642 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.119596 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.120043 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.120199 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.123606 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:07.123612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:07.123615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:07.123649 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:07.127891 (+ 4242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.128728 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.128741 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.128743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.128953 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.129376 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.129379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.129381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.129880 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:07.130132 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.130590 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.130823 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 22:14:07.136093 28925 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:14:07.146184 28925 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:14:07.187727 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.164702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44496 (local address 127.10.173.130:38193)
0811 22:14:07.167355 (+ 2653us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.167365 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.168552 (+ 1187us) server_negotiation.cc:407] Connection header received
0811 22:14:07.168963 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.168977 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.169314 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.169602 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:07.175998 (+ 6396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.179338 (+ 3340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.183812 (+ 4474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.185628 (+ 1816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.185890 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.186155 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.186760 (+ 605us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.187017 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2424,"thread_start_us":2152,"threads_started":1}
I20260811 22:14:07.188647 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.167255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53824 (local address 127.10.173.129:43887)
0811 22:14:07.171678 (+ 4423us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.171687 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.176526 (+ 4839us) server_negotiation.cc:407] Connection header received
0811 22:14:07.176826 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.176832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.177099 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.177338 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:07.177998 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.179681 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.184902 (+ 5221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.186833 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.187029 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.187151 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.187777 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.188039 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4122,"thread_start_us":3817,"threads_started":1}
I20260811 22:14:07.189330 29773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.165198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:38193 (local address 127.0.0.1:44496)
0811 22:14:07.168428 (+ 3230us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.168448 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:07.168714 (+ 266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:07.169616 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:07.169634 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:07.169713 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:07.175339 (+ 5626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.175370 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.179623 (+ 4253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.179629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.183572 (+ 3943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.183592 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.185985 (+ 2393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.185994 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.186291 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.186359 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 22:14:07.188107 (+ 1748us) client_negotiation.cc:240] Negotiation successful
0811 22:14:07.188319 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3046,"thread_start_us":10544,"threads_started":1}
I20260811 22:14:07.189446 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.166929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43887 (local address 127.0.0.1:53824)
0811 22:14:07.176397 (+ 9468us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.176417 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:07.176645 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:07.177320 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:07.177324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:07.177350 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:07.177784 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.177796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.179855 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.179860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.184611 (+ 4751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.184632 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.187395 (+ 2763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.187400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.187588 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.187611 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:14:07.187723 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:14:07.188320 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9250,"mutex_wait_us":100,"thread_start_us":1073,"threads_started":1}
I20260811 22:14:07.191553 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.164124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:40703 (local address 127.0.0.1:43456)
0811 22:14:07.165776 (+ 1652us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.165836 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:07.166060 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:07.174098 (+ 8038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:07.174103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:07.174138 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:07.176312 (+ 2174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.176328 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.179590 (+ 3262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.179598 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.185463 (+ 5865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.185481 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.190188 (+ 4707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.190197 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.190468 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.190502 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:14:07.190596 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:14:07.190739 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1294,"mutex_wait_us":73,"thread_start_us":1724,"threads_started":1}
I20260811 22:14:07.192804 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.164303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43456 (local address 127.10.173.131:40703)
0811 22:14:07.173136 (+ 8833us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.173149 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.173232 (+ 83us) server_negotiation.cc:407] Connection header received
0811 22:14:07.173409 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.173422 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.173775 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.174151 (+ 376us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:07.176650 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.179392 (+ 2742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.186308 (+ 6916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.188794 (+ 2486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.189157 (+ 363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.189409 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.191855 (+ 2446us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.192067 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8495,"thread_start_us":8035,"threads_started":1}
I20260811 22:14:07.194738 29395 tablet_service.cc:1467] Processing CreateTablet for tablet cb3c1330fca343158d0d8f8838411302 (DEFAULT_TABLE table=test-table [id=4f8c1374ef644e058064e3530cbb8361]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:07.194585 29172 tablet_service.cc:1467] Processing CreateTablet for tablet cb3c1330fca343158d0d8f8838411302 (DEFAULT_TABLE table=test-table [id=4f8c1374ef644e058064e3530cbb8361]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:07.196480 29395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb3c1330fca343158d0d8f8838411302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:07.196705 29172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb3c1330fca343158d0d8f8838411302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:07.198091 29636 tablet_service.cc:1467] Processing CreateTablet for tablet cb3c1330fca343158d0d8f8838411302 (DEFAULT_TABLE table=test-table [id=4f8c1374ef644e058064e3530cbb8361]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:14:07.200060 29636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb3c1330fca343158d0d8f8838411302. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:07.213529 29776 tablet_bootstrap.cc:492] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: Bootstrap starting.
I20260811 22:14:07.218896 29777 tablet_bootstrap.cc:492] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8: Bootstrap starting.
I20260811 22:14:07.222525 29776 tablet_bootstrap.cc:654] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:07.223336 29778 tablet_bootstrap.cc:492] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: Bootstrap starting.
I20260811 22:14:07.224905 29777 tablet_bootstrap.cc:654] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:07.226254 29776 log.cc:824] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:07.227975 29777 log.cc:824] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:07.229537 29778 tablet_bootstrap.cc:654] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:07.231644 29776 tablet_bootstrap.cc:492] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: No bootstrap required, opened a new log
I20260811 22:14:07.231779 29778 log.cc:824] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:07.232125 29776 ts_tablet_manager.cc:1397] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.007s
I20260811 22:14:07.236104 29777 tablet_bootstrap.cc:492] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8: No bootstrap required, opened a new log
I20260811 22:14:07.236485 29777 ts_tablet_manager.cc:1397] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.007s
I20260811 22:14:07.242336 29778 tablet_bootstrap.cc:492] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: No bootstrap required, opened a new log
I20260811 22:14:07.242851 29778 ts_tablet_manager.cc:1397] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.008s
I20260811 22:14:07.244083 29776 raft_consensus.cc:348] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c96d0135804a59838c6c8a97bdbf75" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40703 } } peers { permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 } } peers { permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 } }
I20260811 22:14:07.244545 29776 raft_consensus.cc:374] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:07.244671 29776 raft_consensus.cc:729] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50f46af4c37445328ff9a155360e0ca2, State: Initialized, Role: FOLLOWER
I20260811 22:14:07.245342 29776 consensus_queue.cc:260] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c96d0135804a59838c6c8a97bdbf75" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40703 } } peers { permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 } } peers { permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 } }
I20260811 22:14:07.248018 29776 ts_tablet_manager.cc:1428] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: Time spent starting tablet: real 0.016s user 0.008s sys 0.007s
I20260811 22:14:07.251791 29777 raft_consensus.cc:348] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c96d0135804a59838c6c8a97bdbf75" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40703 } } peers { permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 } } peers { permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 } }
I20260811 22:14:07.252204 29777 raft_consensus.cc:374] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:07.252358 29777 raft_consensus.cc:729] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e6191162e4942dcb789f5e036352fa8, State: Initialized, Role: FOLLOWER
I20260811 22:14:07.253125 29777 consensus_queue.cc:260] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c96d0135804a59838c6c8a97bdbf75" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40703 } } peers { permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 } } peers { permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 } }
I20260811 22:14:07.255807 29777 ts_tablet_manager.cc:1428] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8: Time spent starting tablet: real 0.019s user 0.016s sys 0.004s
I20260811 22:14:07.258402 29778 raft_consensus.cc:348] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c96d0135804a59838c6c8a97bdbf75" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40703 } } peers { permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 } } peers { permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 } }
I20260811 22:14:07.259095 29778 raft_consensus.cc:374] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:07.259296 29778 raft_consensus.cc:729] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c96d0135804a59838c6c8a97bdbf75, State: Initialized, Role: FOLLOWER
I20260811 22:14:07.259943 29778 consensus_queue.cc:260] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c96d0135804a59838c6c8a97bdbf75" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40703 } } peers { permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 } } peers { permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 } }
I20260811 22:14:07.261477 29745 heartbeater.cc:503] Master 127.10.173.190:32809 was elected leader, sending a full tablet report...
I20260811 22:14:07.262317 29778 ts_tablet_manager.cc:1428] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: Time spent starting tablet: real 0.019s user 0.013s sys 0.004s
I20260811 22:14:07.264428 29786 raft_consensus.cc:482] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:14:07.264820 29786 raft_consensus.cc:504] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c96d0135804a59838c6c8a97bdbf75" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40703 } } peers { permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 } } peers { permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 } }
I20260811 22:14:07.267961 29786 leader_election.cc:288] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e6191162e4942dcb789f5e036352fa8 (127.10.173.129:43887), 50f46af4c37445328ff9a155360e0ca2 (127.10.173.130:38193)
I20260811 22:14:07.277662 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.268051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:56033 (local address 127.10.173.130:38193)
0811 22:14:07.268308 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.268313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.269349 (+ 1036us) server_negotiation.cc:407] Connection header received
0811 22:14:07.269444 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.269450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.269557 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.269686 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:07.270771 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.272096 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.274855 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.276644 (+ 1789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.276848 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.276942 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.277240 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.277392 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:14:07.277679 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.268823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:38193 (local address 127.10.173.131:56033)
0811 22:14:07.269023 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.269044 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:07.269227 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:07.269758 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:07.269776 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:07.269814 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:07.270298 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.270318 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.272311 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.272315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.274683 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.274696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.276842 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.276847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.277038 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.277063 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:14:07.277181 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:14:07.277311 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":6}
I20260811 22:14:07.279209 29430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb3c1330fca343158d0d8f8838411302" candidate_uuid: "07c96d0135804a59838c6c8a97bdbf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50f46af4c37445328ff9a155360e0ca2" is_pre_election: true
I20260811 22:14:07.279902 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.268466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:54777 (local address 127.10.173.129:43887)
0811 22:14:07.268707 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.268724 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.271725 (+ 3001us) server_negotiation.cc:407] Connection header received
0811 22:14:07.271906 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.271909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.271970 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.272042 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:14:07.272892 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.274384 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.277415 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.278719 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.278842 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.278892 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.279540 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.279723 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:14:07.279955 29430 raft_consensus.cc:2454] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07c96d0135804a59838c6c8a97bdbf75 in term 0.
I20260811 22:14:07.280439 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.269447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43887 (local address 127.10.173.131:54777)
0811 22:14:07.271595 (+ 2148us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.271613 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:07.271766 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:07.272086 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:07.272091 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:07.272116 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:14:07.272719 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.272729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.274619 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.274624 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.277193 (+ 2569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:07.277210 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.279020 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.279028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:07.279334 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.279378 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 22:14:07.279491 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:14:07.279635 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1987,"mutex_wait_us":32,"thread_start_us":1637,"threads_started":1}
I20260811 22:14:07.280900 29550 leader_election.cc:302] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07c96d0135804a59838c6c8a97bdbf75, 50f46af4c37445328ff9a155360e0ca2; no voters:
I20260811 22:14:07.281411 29786 raft_consensus.cc:2781] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:14:07.281569 29786 raft_consensus.cc:482] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:14:07.281702 29786 raft_consensus.cc:3037] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:07.281980 29205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb3c1330fca343158d0d8f8838411302" candidate_uuid: "07c96d0135804a59838c6c8a97bdbf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6191162e4942dcb789f5e036352fa8" is_pre_election: true
I20260811 22:14:07.282761 29205 raft_consensus.cc:2454] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07c96d0135804a59838c6c8a97bdbf75 in term 0.
I20260811 22:14:07.284718 29786 raft_consensus.cc:504] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c96d0135804a59838c6c8a97bdbf75" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40703 } } peers { permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 } } peers { permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 } }
I20260811 22:14:07.285583 29786 leader_election.cc:288] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [CANDIDATE]: Term 1 election: Requested vote from peers 5e6191162e4942dcb789f5e036352fa8 (127.10.173.129:43887), 50f46af4c37445328ff9a155360e0ca2 (127.10.173.130:38193)
I20260811 22:14:07.286182 29430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb3c1330fca343158d0d8f8838411302" candidate_uuid: "07c96d0135804a59838c6c8a97bdbf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50f46af4c37445328ff9a155360e0ca2"
I20260811 22:14:07.286330 29205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb3c1330fca343158d0d8f8838411302" candidate_uuid: "07c96d0135804a59838c6c8a97bdbf75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e6191162e4942dcb789f5e036352fa8"
I20260811 22:14:07.286540 29430 raft_consensus.cc:3037] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:07.286669 29205 raft_consensus.cc:3037] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:07.292200 29430 raft_consensus.cc:2454] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07c96d0135804a59838c6c8a97bdbf75 in term 1.
I20260811 22:14:07.292363 29205 raft_consensus.cc:2454] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07c96d0135804a59838c6c8a97bdbf75 in term 1.
I20260811 22:14:07.292882 29550 leader_election.cc:302] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07c96d0135804a59838c6c8a97bdbf75, 50f46af4c37445328ff9a155360e0ca2; no voters:
I20260811 22:14:07.293355 29786 raft_consensus.cc:2781] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:07.294566 29786 raft_consensus.cc:686] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 1 LEADER]: Becoming Leader. State: Replica: 07c96d0135804a59838c6c8a97bdbf75, State: Running, Role: LEADER
I20260811 22:14:07.295428 29786 consensus_queue.cc:237] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c96d0135804a59838c6c8a97bdbf75" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40703 } } peers { permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 } } peers { permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 } }
I20260811 22:14:07.302145 28925 catalog_manager.cc:5495] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07c96d0135804a59838c6c8a97bdbf75 (127.10.173.131). New cstate: current_term: 1 leader_uuid: "07c96d0135804a59838c6c8a97bdbf75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c96d0135804a59838c6c8a97bdbf75" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 } health_report { overall_health: UNKNOWN } } }
W20260811 22:14:07.311326 29746 tablet.cc:2369] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:14:07.316360 29510 tablet.cc:2369] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:14:07.343022 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.335645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43458 (local address 127.10.173.131:40703)
0811 22:14:07.335930 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.335936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.335959 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:14:07.336209 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.336213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.336292 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.336449 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:07.337174 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.338234 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.340232 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.340897 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.341049 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.341730 (+ 681us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:07.342291 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:07.342537 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.342589 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.342672 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:14:07.353894 29205 raft_consensus.cc:1261] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8 [term 1 FOLLOWER]: Refusing update from remote peer 07c96d0135804a59838c6c8a97bdbf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:07.353844 29430 raft_consensus.cc:1261] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2 [term 1 FOLLOWER]: Refusing update from remote peer 07c96d0135804a59838c6c8a97bdbf75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:07.355304 29791 consensus_queue.cc:1035] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "50f46af4c37445328ff9a155360e0ca2" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 38193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:07.355934 29786 consensus_queue.cc:1035] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e6191162e4942dcb789f5e036352fa8" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:07.373154 29797 mvcc.cc:205] Tried to move back new op lower bound from 7317448488347242496 to 7317448488126869504. Current Snapshot: MvccSnapshot[applied={T|T < 7317448488347242496}]
I20260811 22:14:07.394979 28925 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:07.395354 28925 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:14:07.401018 28925 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 355a35f82f4445e5af0f5142a0129cc4: Sending DeleteTablet for 3 replicas of tablet cb3c1330fca343158d0d8f8838411302
I20260811 22:14:07.402997 29636 tablet_service.cc:1514] Processing DeleteTablet for tablet cb3c1330fca343158d0d8f8838411302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:07 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:43456
I20260811 22:14:07.403040 29172 tablet_service.cc:1514] Processing DeleteTablet for tablet cb3c1330fca343158d0d8f8838411302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:07 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:53824
I20260811 22:14:07.403340 29395 tablet_service.cc:1514] Processing DeleteTablet for tablet cb3c1330fca343158d0d8f8838411302 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:14:07 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:44496
I20260811 22:14:07.406724 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 29064
I20260811 22:14:07.416465 29811 tablet_replica.cc:330] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: stopping tablet replica
I20260811 22:14:07.418612 29811 raft_consensus.cc:2229] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:14:07.420336 29811 raft_consensus.cc:2258] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:07.420868 29813 tablet_replica.cc:330] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: stopping tablet replica
I20260811 22:14:07.421360 29813 raft_consensus.cc:2229] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:07.422024 29813 raft_consensus.cc:2258] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:07.424981 29813 ts_tablet_manager.cc:1905] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:07.426082 29811 ts_tablet_manager.cc:1905] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 22:14:07 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:14:07.431100 29812 tablet_replica.cc:330] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8: stopping tablet replica
I20260811 22:14:07.431799 29812 raft_consensus.cc:2229] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:14:07.432864 29812 raft_consensus.cc:2258] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:14:07.436543 29813 ts_tablet_manager.cc:1918] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:14:07.436892 29813 log.cc:1196] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/cb3c1330fca343158d0d8f8838411302
I20260811 22:14:07.436915 29812 ts_tablet_manager.cc:1905] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:14:07.437983 29813 ts_tablet_manager.cc:1939] T cb3c1330fca343158d0d8f8838411302 P 50f46af4c37445328ff9a155360e0ca2: Deleting consensus metadata
I20260811 22:14:07.440857 28904 catalog_manager.cc:4841] TS 50f46af4c37445328ff9a155360e0ca2 (127.10.173.130:38193): tablet cb3c1330fca343158d0d8f8838411302 (table test-table [id=4f8c1374ef644e058064e3530cbb8361]) successfully deleted
I20260811 22:14:07.443063 29811 ts_tablet_manager.cc:1918] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:14:07.443367 29811 log.cc:1196] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/cb3c1330fca343158d0d8f8838411302
I20260811 22:14:07.444268 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.408109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.10.173.129:43887)
0811 22:14:07.408365 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.408376 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.411836 (+ 3460us) server_negotiation.cc:407] Connection header received
0811 22:14:07.412075 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.412082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.412206 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.412337 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:07.418022 (+ 5685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.419325 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.421524 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.422656 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.422781 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.431500 (+ 8719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:07.431521 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:07.431556 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:07.431754 (+ 198us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:07.437976 (+ 6222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.439482 (+ 1506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.439493 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.439497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.439756 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.440591 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.440597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.440600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.441570 (+ 970us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:07.442065 (+ 495us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.442979 (+ 914us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.443533 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:14:07.444423 29811 ts_tablet_manager.cc:1939] T cb3c1330fca343158d0d8f8838411302 P 07c96d0135804a59838c6c8a97bdbf75: Deleting consensus metadata
I20260811 22:14:07.446748 29269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:14:07.447501 28909 catalog_manager.cc:4841] TS 07c96d0135804a59838c6c8a97bdbf75 (127.10.173.131:40703): tablet cb3c1330fca343158d0d8f8838411302 (table test-table [id=4f8c1374ef644e058064e3530cbb8361]) successfully deleted
I20260811 22:14:07.547626 29812 ts_tablet_manager.cc:1918] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:14:07.548071 29812 log.cc:1196] T cb3c1330fca343158d0d8f8838411302 P 5e6191162e4942dcb789f5e036352fa8: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/cb3c1330fca343158d0d8f8838411302
I20260811 22:14:07.548336 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 29064
W20260811 22:14:07.583801 28909 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.173.129:43887 (error 108)
I20260811 22:14:07.584734 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 29282
I20260811 22:14:07.586252 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.585491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43887 (local address 127.0.0.1:53854)
0811 22:14:07.585921 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.586051 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281}
W20260811 22:14:07.586606 28909 catalog_manager.cc:4584] TS 5e6191162e4942dcb789f5e036352fa8 (127.10.173.129:43887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb3c1330fca343158d0d8f8838411302: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43887: connect: Connection refused (error 111)
Aug 11 22:14:07 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:14:07.604045 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.585752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44506 (local address 127.10.173.130:38193)
0811 22:14:07.585976 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.585985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.586099 (+ 114us) server_negotiation.cc:407] Connection header received
0811 22:14:07.586323 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.586330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.586449 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.586569 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:07.588125 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.589312 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.591034 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.591692 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.591830 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.596752 (+ 4922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:07.596774 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:07.596797 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:07.596843 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:07.600798 (+ 3955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.601616 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.601628 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.601632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.601799 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.602208 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.602213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.602217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.602736 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:07.603055 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.603333 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.603601 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:14:07.604923 29498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:14:07.640707 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.640067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43887 (local address 127.0.0.1:53866)
0811 22:14:07.640343 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.640480 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 22:14:07.642042 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.641597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43887 (local address 127.0.0.1:53878)
0811 22:14:07.641789 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.641898 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:14:07.698503 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 29282
I20260811 22:14:07.703783 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.703256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43887 (local address 127.0.0.1:53882)
0811 22:14:07.703472 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.703580 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 22:14:07.704865 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.704513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43887 (local address 127.0.0.1:53894)
0811 22:14:07.704674 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.704755 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:14:07.732949 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 29518
Aug 11 22:14:07 dist-test-slave-68xk krb5kdc[28149](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:14:07.752234 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.733904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43472 (local address 127.10.173.131:40703)
0811 22:14:07.734101 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.734106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.734151 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:14:07.734377 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.734382 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.734509 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.734633 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:07.736549 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.737361 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.739411 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.739879 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.739978 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.745785 (+ 5807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:07.745798 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:07.745813 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:07.745851 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:07.749164 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.750099 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.750107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.750109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.750254 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.750728 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.750732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.750734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.751104 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:07.751366 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.751761 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.751926 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:14:07.753082 29739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:14:07.785753 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.785165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43887 (local address 127.0.0.1:53910)
0811 22:14:07.785439 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.785540 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 22:14:07.787170 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.786642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43887 (local address 127.0.0.1:53926)
0811 22:14:07.786930 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:07.787027 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 22:14:07.893584 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 29518
I20260811 22:14:07.928934 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 28880
I20260811 22:14:07.930284 28982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:14:08.048623 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 28880
I20260811 22:14:08.048707 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:08.047931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43887 (local address 127.0.0.1:53932)
0811 22:14:08.048157 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:08.048327 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 22:14:08.081609 10934 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:14:08.427410 10934 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:14:08.427747 10934 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:14:08.783691 10934 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:14:08.784 UTC [28191] LOG: received smart shutdown request
2026-08-11 22:14:08.788 UTC [28191] LOG: background worker "logical replication launcher" (PID 28204) exited with exit code 1
2026-08-11 22:14:08.789 UTC [28198] LOG: shutting down
2026-08-11 22:14:08.984 UTC [28191] LOG: database system is shut down
2026-08-11T22:14:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54659 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:09 dist-test-slave-68xk krb5kdc[29833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:09 dist-test-slave-68xk krb5kdc[29833](info): set up 2 sockets
Aug 11 22:14:09 dist-test-slave-68xk krb5kdc[29833](info): commencing operation
krb5kdc: starting...
W20260811 22:14:10.480682 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:14:10 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.173.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.173.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/rangeradmin_127.10.173.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/rangeradmin_127.10.173.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.173.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.173.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/rangerlookup_127.10.173.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/rangerlookup_127.10.173.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.173.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.173.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/HTTP_127.10.173.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/HTTP_127.10.173.148@KRBTEST.COM.keytab.
I20260811 22:14:10.887212 10934 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskZXazun/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/postgres -l logfile start
2026-08-11 22:14:13.604 UTC [29886] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:14:13.604 UTC [29886] LOG: listening on IPv4 address "127.10.173.148", port 42647
2026-08-11 22:14:13.608 UTC [29886] LOG: listening on Unix socket "/tmp/.s.PGSQL.42647"
2026-08-11 22:14:13.634 UTC [29890] LOG: database system was shut down at 2026-08-11 22:14:11 UTC
2026-08-11 22:14:13.638 UTC [29886] LOG: database system is ready to accept connections
I20260811 22:14:14.842530 10934 mini_postgres.cc:96] Postgres bound to 42647
2026-08-11 22:14:14.865 UTC [29905] FATAL: database "slave" does not exist
127.10.173.148:42647 - accepting connections
I20260811 22:14:14.866632 10934 mini_ranger.cc:161] Starting Ranger...
I20260811 22:14:14.897475 10934 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:14:15.459218 10934 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:14:15.459344 10934 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-admin
2026-08-11 22:14:15,779 [I] DB FLAVOR :POSTGRES
2026-08-11 22:14:15,779 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:14:15,779 [I] Checking connection..
2026-08-11 22:14:15,780 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:14:16,050 [I] Checking connection passed.
2026-08-11 22:14:16,051 [I] --------- Verifying version history table ---------
2026-08-11 22:14:16,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:14:16,303 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:14:16,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:14:16,552 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:14:16,553 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:14:16,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:14:16.791 UTC [30023] WARNING: there is no transaction in progress
2026-08-11 22:14:16,798 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:14:16,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:14:17,053 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:14:17,053 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:14:17,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:14:17,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:14:17,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-68xk.c.gcp-upstream.internal','N') ;"
2026-08-11 22:14:17,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:14:18,109 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:14:18,109 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:14:18,368 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:14:18,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:14:18,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:14:18,910 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:14:18,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:14:19.974 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:19.990 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.009 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.028 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.046 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.065 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.128 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.135 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.151 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.161 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.171 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.186 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.196 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.203 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.209 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20.216 UTC [30202] WARNING: there is no transaction in progress
2026-08-11 22:14:20,952 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:14:20,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:14:21,203 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:14:21,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:14:21,448 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:14:21,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:14:21,727 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:14:21,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:14:22,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:14:22,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-68xk.c.gcp-upstream.internal';"
2026-08-11 22:14:22,589 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:14:22,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:14:22,855 [I] DB_PATCHES have already been applied
I20260811 22:14:22.861871 10934 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-admin:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskZXazun/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:14:22.861968 10934 mini_ranger.cc:193] Using host: 127.10.173.148
I20260811 22:14:22.882520 10934 mini_ranger.cc:227] Ranger admin URL: http://127.10.173.148:38747
Aug 11, 2026 10:14:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:14:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:14:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:14:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:14:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:14:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:14:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:14:24.326308 10934 mini_ranger.cc:160] Time spent starting Ranger: real 9.460s user 0.003s sys 0.055s
Aug 11, 2026 10:14:44 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:14:44 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 10:14:44 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:14:44 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:14:45 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:14:45.577781 10934 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.173.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.173.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/rangerkms_127.10.173.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.173.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/rangerkms_127.10.173.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.173.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.173.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.173.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/HTTP_127.10.173.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.173.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/HTTP_127.10.173.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:14:45 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486485, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:14:45.786247 10934 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:14:45.812839 10934 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:14:46.794304 10934 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:14:46.794476 10934 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-kms
I20260811 22:14:46.794564 10934 mini_ranger_kms.cc:227] Using postgres at 127.10.173.148:42647
2026-08-11 22:14:47,195 [I] DB FLAVOR :POSTGRES
2026-08-11 22:14:47,196 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:14:47,196 [I] Checking connection
2026-08-11 22:14:47,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:14:47,470 [I] connection success
2026-08-11 22:14:47,470 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:14:47,470 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:14:47,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:14:47,737 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:14:47,738 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:14:47,738 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:14:47,738 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.173.148:42647/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:14:48,024 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:14:48.512797 10934 mini_ranger_kms.cc:310] Created kms service
I20260811 22:14:48.685792 10934 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:14:48.734140 10934 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:14:48 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486485, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:49.127115 10934 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:14:49.127486 10934 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-kms:/tmp/dist-test-taskZXazun/build/asan/bin/postgresql.jar:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskZXazun/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:14:49.127558 10934 mini_ranger_kms.cc:240] Using host: 127.10.173.148
I20260811 22:14:49.147989 10934 mini_ranger_kms.cc:276] Ranger KMS PID: 30501
I20260811 22:14:49.148379 10934 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.173.148:38157
Aug 11, 2026 10:14:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:14:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:14:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:14:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:14:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:14:50 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 22:14:51 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486491, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.173.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 22:14:52 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486491, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.173.148@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:52.062556 10934 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.276s user 0.066s sys 0.367s
I20260811 22:14:52.062844 10934 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:14:52.063024 10934 mini_ranger_kms.cc:401] 127.10.173.148:38157/kms/v1/keys
Aug 11 22:14:52 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486491, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.173.148@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
Aug 11 22:14:54 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:14:54.819243 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41969
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:41969
--ranger_config_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:38157/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:55.178812 30600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:55.179180 30600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:55.179343 30600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:55.186839 30600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:55.186987 30600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:55.187091 30600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:55.187204 30600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:55.187296 30600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:55.197933 30600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.173.148:38157/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:41969
--ranger_config_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:14:55.199380 30600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:55.200762 30600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:55.213581 30612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:55.213550 30606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:55.213657 30609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:55.214385 30600 server_base.cc:1034] running on GCE node
I20260811 22:14:55.215176 30600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:55.216598 30600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:55.217837 30600 hybrid_clock.cc:648] HybridClock initialized: now 1786486495217797 us; error 59 us; skew 500 ppm
I20260811 22:14:55.218205 30600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:14:55 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:55.221612 30600 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:14:55.223281 30600 webserver.cc:460] Webserver started at http://127.10.173.190:45893/ using document root <none> and password file <none>
I20260811 22:14:55.223912 30600 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:55.224020 30600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:55.224306 30600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:14:55 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486495, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:55.286775 30600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "db073fe734684b88af8a86180428d125"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-68xk"
server_key: "2e887a77eec30d294cee4275aa403ef8"
server_key_iv: "ad7eee0278bcd1010699210e01cd578a"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:55.288113 30600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "db073fe734684b88af8a86180428d125"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-68xk"
server_key: "2e887a77eec30d294cee4275aa403ef8"
server_key_iv: "ad7eee0278bcd1010699210e01cd578a"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:55.295636 30600 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 22:14:55.317260 30629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:55.319124 30600 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:55.319346 30600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "db073fe734684b88af8a86180428d125"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-68xk"
server_key: "2e887a77eec30d294cee4275aa403ef8"
server_key_iv: "ad7eee0278bcd1010699210e01cd578a"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:55.319621 30600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:55.359508 30600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:55.363950 30600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:55.364310 30600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:55.384156 30600 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:41969
I20260811 22:14:55.384181 30724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:41969 every 8 connection(s)
I20260811 22:14:55.385643 30600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:55.389770 30725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:55.390859 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 30600
I20260811 22:14:55.391098 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:14:55.392020 10934 external_mini_cluster.cc:1428] Setting key 04a2505dc4e9270366c4685f806a14d2
I20260811 22:14:55.406143 30725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: Bootstrap starting.
Aug 11 22:14:55 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:14:55.410735 30725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:55.412336 30725 log.cc:824] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:55.416373 30725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: No bootstrap required, opened a new log
I20260811 22:14:55.420594 30729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:55.394320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.10.173.190:41969)
0811 22:14:55.395369 (+ 1049us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:55.395381 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:55.395433 (+ 52us) server_negotiation.cc:407] Connection header received
0811 22:14:55.397246 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:55.397271 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:55.397727 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:55.398092 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:55.400049 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.402225 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.404403 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:55.405220 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:55.405376 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:55.410840 (+ 5464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:55.410855 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:55.410877 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:55.410933 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:55.415331 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.416267 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.416277 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.416286 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.416518 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:55.416969 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:55.416974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:55.416977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:55.417795 (+ 818us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:55.418178 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:55.418539 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 22:14:55.419065 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":503,"threads_started":1}
I20260811 22:14:55.427325 30725 raft_consensus.cc:348] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } }
I20260811 22:14:55.427736 30725 raft_consensus.cc:374] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:55.427904 30725 raft_consensus.cc:729] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db073fe734684b88af8a86180428d125, State: Initialized, Role: FOLLOWER
I20260811 22:14:55.428612 30725 consensus_queue.cc:260] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } }
I20260811 22:14:55.428838 30725 raft_consensus.cc:388] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:55.428946 30725 raft_consensus.cc:482] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:55.429100 30725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:55.431721 30725 raft_consensus.cc:504] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } }
I20260811 22:14:55.432322 30725 leader_election.cc:302] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db073fe734684b88af8a86180428d125; no voters:
I20260811 22:14:55.433032 30725 leader_election.cc:288] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:55.433240 30732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:55.433995 30732 raft_consensus.cc:686] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 1 LEADER]: Becoming Leader. State: Replica: db073fe734684b88af8a86180428d125, State: Running, Role: LEADER
I20260811 22:14:55.434723 30732 consensus_queue.cc:237] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } }
I20260811 22:14:55.435513 30725 sys_catalog.cc:564] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:55.438604 30737 sys_catalog.cc:455] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db073fe734684b88af8a86180428d125. Latest consensus state: current_term: 1 leader_uuid: "db073fe734684b88af8a86180428d125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } } }
I20260811 22:14:55.439175 30737 sys_catalog.cc:458] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:55.438616 30735 sys_catalog.cc:455] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db073fe734684b88af8a86180428d125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } } }
I20260811 22:14:55.439837 30735 sys_catalog.cc:458] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:55.443452 30742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:55.448408 30742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:55.453599 30725 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:14:55.458611 30742 catalog_manager.cc:1347] Generated new cluster ID: dbd3e117c41d4cdbbb3a352cab6d90c3
I20260811 22:14:55.458809 30742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:55.485483 30742 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:55.488449 30742 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:55.507680 30742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: Generated new TSK 0
I20260811 22:14:55.508704 30742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:14:56.319010 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:38157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41969
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:14:56 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486496, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:14:56.750944 30796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:56.751397 30796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:56.751570 30796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:56.763557 30796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:56.763710 30796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:56.763981 30796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:14:56.780215 30796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.173.148:38157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:14:56.782205 30796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:56.784109 30796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:56.802580 30817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:56.802772 30818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:56.804260 30825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:56.807672 30796 server_base.cc:1034] running on GCE node
I20260811 22:14:56.808631 30796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:56.816114 30796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:56.817728 30796 hybrid_clock.cc:648] HybridClock initialized: now 1786486496817596 us; error 132 us; skew 500 ppm
I20260811 22:14:56.818257 30796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:14:56 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486496, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:56.823464 30796 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:14:56.828740 30796 webserver.cc:460] Webserver started at http://127.10.173.129:40677/ using document root <none> and password file <none>
I20260811 22:14:56.829836 30796 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:56.830015 30796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:56.830521 30796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:14:56 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486496, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:56.855374 30796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "a8152657189f43b4b1b009c3cc4abd89"
format_stamp: "Formatted at 2026-08-11 22:14:56 on dist-test-slave-68xk"
server_key: "31c22ab832624bd44a31125df769f7fb"
server_key_iv: "aaf6928791d86ba8509518b3a8bf76a0"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:56.858641 30796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "a8152657189f43b4b1b009c3cc4abd89"
format_stamp: "Formatted at 2026-08-11 22:14:56 on dist-test-slave-68xk"
server_key: "31c22ab832624bd44a31125df769f7fb"
server_key_iv: "aaf6928791d86ba8509518b3a8bf76a0"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:56.881170 30796 fs_manager.cc:691] Time spent creating directory manager: real 0.022s user 0.008s sys 0.011s
I20260811 22:14:56.923246 30835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:56.926226 30796 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.011s sys 0.007s
I20260811 22:14:56.926560 30796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "a8152657189f43b4b1b009c3cc4abd89"
format_stamp: "Formatted at 2026-08-11 22:14:56 on dist-test-slave-68xk"
server_key: "31c22ab832624bd44a31125df769f7fb"
server_key_iv: "aaf6928791d86ba8509518b3a8bf76a0"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:56.926928 30796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:56.953127 30796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:56.957926 30796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:56.958293 30796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:56.959460 30796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:56.961151 30796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:56.961279 30796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:56.961551 30796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:56.961665 30796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:14:57 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486496, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:57.297143 30796 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:45841
I20260811 22:14:57.299407 30796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:14:57.297183 31020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:45841 every 8 connection(s)
I20260811 22:14:57.304128 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 30796
I20260811 22:14:57.304371 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:14:57.305269 10934 external_mini_cluster.cc:1428] Setting key 1be80092184861fe601b3877dd43ddd1
Aug 11 22:14:57 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486496, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:14:57.364341 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:57.306756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:37013 (local address 127.10.173.190:41969)
0811 22:14:57.334486 (+ 27730us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:57.334493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:57.334520 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:14:57.334596 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:57.334602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:57.334718 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:57.334927 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:57.337252 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.338496 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.350632 (+ 12136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.351355 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.351472 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:57.356635 (+ 5163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:57.356643 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:57.356648 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:57.356704 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:57.360450 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:57.361487 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:57.361493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:57.361496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:57.361656 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:57.362214 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:57.362219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:57.362223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:57.362599 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:57.362716 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:57.363726 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0811 22:14:57.363950 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27542,"thread_start_us":27419,"threads_started":1}
I20260811 22:14:57.366866 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:57.306055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41969 (local address 127.10.173.129:37013)
0811 22:14:57.320706 (+ 14651us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:57.320768 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:57.323493 (+ 2725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:57.335320 (+ 11827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:57.335337 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:57.336053 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:57.336983 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:57.337011 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.338737 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.338743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:57.340787 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:57.340810 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:57.351907 (+ 11097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:57.351915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:57.352288 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:57.353223 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:57.353256 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:57.356313 (+ 3057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:57.360675 (+ 4362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:57.360685 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:57.360704 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:57.361336 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:57.361805 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:57.361809 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:57.361812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:57.362068 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:57.362824 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:57.362832 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:57.363297 (+ 465us) client_negotiation.cc:769] Sending connection context
0811 22:14:57.364837 (+ 1540us) client_negotiation.cc:240] Negotiation successful
0811 22:14:57.365439 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14251,"thread_start_us":17793,"threads_started":1}
I20260811 22:14:57.369012 31023 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41969
I20260811 22:14:57.369349 31023 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:57.370203 31023 heartbeater.cc:511] Master 127.10.173.190:41969 requested a full tablet report, sending...
I20260811 22:14:57.372836 30656 ts_manager.cc:194] Registered new tserver with Master: a8152657189f43b4b1b009c3cc4abd89 (127.10.173.129:45841)
I20260811 22:14:57.375001 30656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:37013
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
W20260811 22:14:57.442135 10934 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.10.173.130: real 0.131s user 0.003s sys 0.022s
I20260811 22:14:57.443202 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:38157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41969
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:14:57 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486496, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
W20260811 22:14:57.933080 31029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:57.933379 31029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:57.933533 31029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:57.941000 31029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:57.941125 31029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:57.941295 31029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:14:57.951453 31029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.173.148:38157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:14:57.952857 31029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:57.954226 31029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:57.969101 31051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:57.969082 31053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:57.969897 31067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:57.970016 31029 server_base.cc:1034] running on GCE node
I20260811 22:14:57.970788 31029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:57.971892 31029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:57.973156 31029 hybrid_clock.cc:648] HybridClock initialized: now 1786486497973080 us; error 42 us; skew 500 ppm
I20260811 22:14:57.973490 31029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:14:57 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486497, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:57.976809 31029 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:14:57.978502 31029 webserver.cc:460] Webserver started at http://127.10.173.130:37657/ using document root <none> and password file <none>
I20260811 22:14:57.979250 31029 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:57.979384 31029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:57.979729 31029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:14:57 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486497, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:57.997385 31029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "38566f6eb374433e92ceb023e116438d"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-68xk"
server_key: "9031440ab9cf259678c790f3945d5c43"
server_key_iv: "6cfe95460adbc4cb45c0a726d9aa9aca"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:57.998870 31029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "38566f6eb374433e92ceb023e116438d"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-68xk"
server_key: "9031440ab9cf259678c790f3945d5c43"
server_key_iv: "6cfe95460adbc4cb45c0a726d9aa9aca"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:58.007292 31029 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.002s sys 0.004s
I20260811 22:14:58.028373 31076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.030269 31029 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:58.030507 31029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "38566f6eb374433e92ceb023e116438d"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-68xk"
server_key: "9031440ab9cf259678c790f3945d5c43"
server_key_iv: "6cfe95460adbc4cb45c0a726d9aa9aca"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:58.030814 31029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:58.053771 31029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:58.059173 31029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:58.059511 31029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:58.060786 31029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:58.062690 31029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:58.062790 31029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.062970 31029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:58.063059 31029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.106945 31029 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:42139
I20260811 22:14:58.106990 31261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:42139 every 8 connection(s)
I20260811 22:14:58.108769 31029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:14:58.116541 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 31029
I20260811 22:14:58.116719 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:14:58.117341 10934 external_mini_cluster.cc:1428] Setting key ba1b6e2093e50fbc52edbad9be777669
Aug 11 22:14:58 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486497, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:14:58.155071 31267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.111880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:39137 (local address 127.10.173.190:41969)
0811 22:14:58.130513 (+ 18633us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.130519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.130541 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:14:58.130604 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.130610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.130692 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.130813 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:58.133674 (+ 2861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.134943 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.137433 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.138117 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.138260 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.142627 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:58.142635 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:58.142640 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:58.142686 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:58.148258 (+ 5572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.149352 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.149360 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.149367 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.149533 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.150138 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.150146 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.150152 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.150620 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:58.150781 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.154612 (+ 3831us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.154810 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18450,"thread_start_us":18286,"threads_started":1}
I20260811 22:14:58.155593 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.112465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41969 (local address 127.10.173.130:39137)
0811 22:14:58.126064 (+ 13599us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.126147 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:58.129657 (+ 3510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:58.131142 (+ 1485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:58.131158 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:58.131870 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:58.132766 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.132796 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.135113 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.135119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.137197 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.137214 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.138371 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.138381 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.138772 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.139683 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:58.139716 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:58.142360 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:58.148447 (+ 6087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:58.148457 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:58.148476 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:58.149201 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:58.149679 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:58.149684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:58.149688 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:58.149993 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:58.151077 (+ 1084us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:58.151089 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:58.151621 (+ 532us) client_negotiation.cc:769] Sending connection context
0811 22:14:58.152594 (+ 973us) client_negotiation.cc:240] Negotiation successful
0811 22:14:58.153235 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13207,"thread_start_us":13872,"threads_started":1}
I20260811 22:14:58.157222 31263 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41969
I20260811 22:14:58.157553 31263 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
I20260811 22:14:58.158353 31263 heartbeater.cc:511] Master 127.10.173.190:41969 requested a full tablet report, sending...
I20260811 22:14:58.160020 30656 ts_manager.cc:194] Registered new tserver with Master: 38566f6eb374433e92ceb023e116438d (127.10.173.130:42139)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
I20260811 22:14:58.160895 30656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:39137
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:14:58.201418 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:38157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41969
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:14:58.378942 31023 heartbeater.cc:503] Master 127.10.173.190:41969 was elected leader, sending a full tablet report...
W20260811 22:14:58.596529 31273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:58.596860 31273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:58.597035 31273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:58.605178 31273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:58.605302 31273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:58.605480 31273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:14:58.615581 31273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.173.148:38157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:14:58.617005 31273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:58.618337 31273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:58.632750 31282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:58.632644 31285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:58.632683 31283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:58.633978 31273 server_base.cc:1034] running on GCE node
I20260811 22:14:58.634865 31273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:58.636087 31273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:58.637403 31273 hybrid_clock.cc:648] HybridClock initialized: now 1786486498637327 us; error 53 us; skew 500 ppm
I20260811 22:14:58.637809 31273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:14:58 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486498, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:58.641446 31273 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:14:58.643075 31273 webserver.cc:460] Webserver started at http://127.10.173.131:44085/ using document root <none> and password file <none>
I20260811 22:14:58.643846 31273 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:58.644001 31273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:58.644344 31273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:14:58 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486498, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:58.660797 31273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "ed3ad898b70542439ec8f7f5763e8c6b"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-68xk"
server_key: "0a086db175b13bcabf31a278329534ab"
server_key_iv: "172767622759cbe23ff7788563f98adc"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:58.661753 31273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "ed3ad898b70542439ec8f7f5763e8c6b"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-68xk"
server_key: "0a086db175b13bcabf31a278329534ab"
server_key_iv: "172767622759cbe23ff7788563f98adc"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:58.668118 31273 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 22:14:58.684098 31298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.685892 31273 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:58.686112 31273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "ed3ad898b70542439ec8f7f5763e8c6b"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-68xk"
server_key: "0a086db175b13bcabf31a278329534ab"
server_key_iv: "172767622759cbe23ff7788563f98adc"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:58.686379 31273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:58.705595 31273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:58.709676 31273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:58.709998 31273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:58.711061 31273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:14:58.712587 31273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:14:58.712694 31273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.712847 31273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:14:58.712921 31273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:58.749359 31273 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:44253
I20260811 22:14:58.749545 31494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:44253 every 8 connection(s)
I20260811 22:14:58.750876 31273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:14:58.755584 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 31273
I20260811 22:14:58.755781 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:14:58.756299 10934 external_mini_cluster.cc:1428] Setting key 2022479b5f9b11e0951b885218bf1e81
Aug 11 22:14:58 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486498, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:14:58.772585 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.753591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:53863 (local address 127.10.173.190:41969)
0811 22:14:58.754079 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.754085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.755162 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 22:14:58.757217 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.757222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.757293 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.757435 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:58.759089 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.760009 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.761662 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.762092 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.762260 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.765985 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:58.765990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:58.765993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:58.766062 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:58.768939 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.769700 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.769704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.769707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.769831 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.770565 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.770579 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.770581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.770909 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:58.771121 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.771885 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.772205 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":239,"threads_started":1}
I20260811 22:14:58.773622 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.754205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41969 (local address 127.10.173.131:53863)
0811 22:14:58.754982 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:58.755020 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:58.756890 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:58.757707 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:58.757736 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:58.758160 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:58.758918 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.758935 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.760163 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.760167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.761501 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:58.761513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.762372 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.762377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:58.762699 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.763404 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:58.763435 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:58.765766 (+ 2331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:58.769090 (+ 3324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:58.769097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:58.769112 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:58.769580 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:58.770059 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:58.770064 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:58.770067 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:58.770379 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:58.771066 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:58.771074 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:58.771460 (+ 386us) client_negotiation.cc:769] Sending connection context
0811 22:14:58.771751 (+ 291us) client_negotiation.cc:240] Negotiation successful
0811 22:14:58.772321 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":335,"threads_started":1}
I20260811 22:14:58.775245 31495 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41969
I20260811 22:14:58.775529 31495 heartbeater.cc:464] Registering TS with master...
I20260811 22:14:58.776221 31495 heartbeater.cc:511] Master 127.10.173.190:41969 requested a full tablet report, sending...
I20260811 22:14:58.777667 30656 ts_manager.cc:194] Registered new tserver with Master: ed3ad898b70542439ec8f7f5763e8c6b (127.10.173.131:44253)
I20260811 22:14:58.778631 30656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:53863
I20260811 22:14:58.785086 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:14:58.786873 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 30796
Aug 11 22:14:58 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:14:58.804376 31502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.787861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.10.173.129:45841)
0811 22:14:58.788633 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.788639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.788665 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:14:58.788722 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.788728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.788927 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.789054 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:58.790714 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.791957 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.793352 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.793968 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.794086 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.798601 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:58.798614 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:58.798630 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:58.798667 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:58.801469 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.802175 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.802183 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.802186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.802375 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.802833 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.802837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.802839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.803206 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:58.803481 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.803681 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.803904 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":448,"threads_started":1}
I20260811 22:14:58.805719 31012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:14:58.889591 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 30796
I20260811 22:14:58.910957 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 31029
Aug 11 22:14:58 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:14:58.934270 31505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:58.912038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53580 (local address 127.10.173.130:42139)
0811 22:14:58.912646 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:58.912653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:58.912678 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:14:58.912741 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:58.912747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:58.912949 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:58.913098 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:58.914524 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.915534 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.917134 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:58.917720 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:58.917834 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:58.928146 (+ 10312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:58.928163 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:58.928180 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:58.928234 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:58.931503 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.932362 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.932370 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.932372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.932521 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:58.932861 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:58.932864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:58.932866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:58.933237 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:58.933462 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:58.933744 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 22:14:58.933966 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":342,"threads_started":1}
I20260811 22:14:58.935686 31256 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:14:59.026759 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 31029
I20260811 22:14:59.048653 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 31273
Aug 11 22:14:59 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:14:59.065882 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:59.049755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51376 (local address 127.10.173.131:44253)
0811 22:14:59.050492 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:59.050500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:59.050533 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:14:59.050600 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:59.050607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:59.050830 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:59.051006 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:59.052387 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:59.053414 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:59.054796 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:59.055360 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:59.055507 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:59.059919 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:59.059931 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:59.059947 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:59.059981 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:59.062973 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:59.063669 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:59.063677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:59.063680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:59.063813 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:59.064204 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:59.064209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:59.064213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:59.064570 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:59.064920 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:59.065225 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 22:14:59.065507 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":339,"threads_started":1}
I20260811 22:14:59.067390 31486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:14:59.152101 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 31273
I20260811 22:14:59.178045 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 30600
I20260811 22:14:59.179553 30714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:14:59.291085 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 30600
I20260811 22:14:59.321399 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41969
--webserver_interface=127.10.173.190
--webserver_port=45893
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:41969
--ranger_config_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:38157/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.173.148:38157/kms with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:59.660966 31511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:59.661263 31511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:59.661432 31511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:59.668747 31511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:59.668860 31511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:59.668955 31511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:59.669055 31511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:59.669130 31511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:59.679716 31511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.173.148:38157/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:41969
--ranger_config_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=45893
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:14:59.681077 31511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:59.682519 31511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:59.695221 31519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:59.695137 31521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:59.695183 31523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:59.695971 31511 server_base.cc:1034] running on GCE node
I20260811 22:14:59.696825 31511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:59.698349 31511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:59.699617 31511 hybrid_clock.cc:648] HybridClock initialized: now 1786486499699579 us; error 40 us; skew 500 ppm
I20260811 22:14:59.700006 31511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:14:59 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486499, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:59.703343 31511 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:14:59.704941 31511 webserver.cc:460] Webserver started at http://127.10.173.190:45893/ using document root <none> and password file <none>
I20260811 22:14:59.705592 31511 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:59.705715 31511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:14:59 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486499, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:14:59.792062 31511 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:59.795485 31535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:59.797037 31511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:14:59.797255 31511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "db073fe734684b88af8a86180428d125"
format_stamp: "Formatted at 2026-08-11 22:14:55 on dist-test-slave-68xk"
server_key: "2e887a77eec30d294cee4275aa403ef8"
server_key_iv: "ad7eee0278bcd1010699210e01cd578a"
server_key_version: "kuduclusterkey@0"
I20260811 22:14:59.798247 31511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:59.827591 31511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:59.831913 31511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:59.832233 31511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:59.852051 31511 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:41969
I20260811 22:14:59.852062 31616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:41969 every 8 connection(s)
I20260811 22:14:59.853567 31511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:59.860436 31617 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:14:59.861354 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 31511
I20260811 22:14:59.862514 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:38157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:45841
--local_ip_for_outbound_sockets=127.10.173.129
--tserver_master_addrs=127.10.173.190:41969
--webserver_port=40677
--webserver_interface=127.10.173.129
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.173.148:38157/kms with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:14:59.865514 31617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: Bootstrap starting.
I20260811 22:14:59.887727 31617 log.cc:824] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:59.897158 31617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:14:59.897719 31617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: Bootstrap complete.
I20260811 22:14:59.907502 31617 raft_consensus.cc:348] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } }
I20260811 22:14:59.908341 31617 raft_consensus.cc:729] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: db073fe734684b88af8a86180428d125, State: Initialized, Role: FOLLOWER
I20260811 22:14:59.909021 31617 consensus_queue.cc:260] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } }
I20260811 22:14:59.909252 31617 raft_consensus.cc:388] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:59.909374 31617 raft_consensus.cc:482] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:59.909561 31617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:14:59.913398 31617 raft_consensus.cc:504] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } }
I20260811 22:14:59.913934 31617 leader_election.cc:302] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db073fe734684b88af8a86180428d125; no voters:
I20260811 22:14:59.914659 31617 leader_election.cc:288] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:14:59.914913 31626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:14:59.916136 31626 raft_consensus.cc:686] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [term 2 LEADER]: Becoming Leader. State: Replica: db073fe734684b88af8a86180428d125, State: Running, Role: LEADER
I20260811 22:14:59.916795 31626 consensus_queue.cc:237] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } }
I20260811 22:14:59.916986 31617 sys_catalog.cc:564] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:59.919691 31629 sys_catalog.cc:455] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "db073fe734684b88af8a86180428d125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } } }
I20260811 22:14:59.920146 31629 sys_catalog.cc:458] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:59.920650 31630 sys_catalog.cc:455] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db073fe734684b88af8a86180428d125. Latest consensus state: current_term: 2 leader_uuid: "db073fe734684b88af8a86180428d125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db073fe734684b88af8a86180428d125" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41969 } } }
I20260811 22:14:59.921113 31630 sys_catalog.cc:458] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:59.925593 31640 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:59.933192 31640 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:59.935041 31640 catalog_manager.cc:1259] Loaded cluster ID: dbd3e117c41d4cdbbb3a352cab6d90c3
I20260811 22:14:59.935384 31640 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:59.938799 31617 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:14:59.940150 31640 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:59.942094 31640 catalog_manager.cc:5879] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: Loaded TSK: 0
I20260811 22:14:59.944046 31640 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:15:00.344430 31620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:00.344853 31620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:00.345104 31620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:00.357080 31620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:00.357292 31620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:00.357532 31620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:15:00.373764 31620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.173.148:38157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:45841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=40677
--tserver_master_addrs=127.10.173.190:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:00.375821 31620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:00.377730 31620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:00.399446 31680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:00.400661 31681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:00.408181 31687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:00.408728 31620 server_base.cc:1034] running on GCE node
I20260811 22:15:00.409766 31620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:00.417273 31620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:00.420804 31620 hybrid_clock.cc:648] HybridClock initialized: now 1786486500420708 us; error 94 us; skew 500 ppm
I20260811 22:15:00.421345 31620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:00 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486500, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:00.426359 31620 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:15:00.452190 31620 webserver.cc:460] Webserver started at http://127.10.173.129:40677/ using document root <none> and password file <none>
I20260811 22:15:00.453465 31620 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:00.453672 31620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:15:00 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486500, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:15:00.502357 31620 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.002s sys 0.007s
I20260811 22:15:00.512946 31697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:00.515408 31620 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.010s sys 0.001s
I20260811 22:15:00.515643 31620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "a8152657189f43b4b1b009c3cc4abd89"
format_stamp: "Formatted at 2026-08-11 22:14:56 on dist-test-slave-68xk"
server_key: "31c22ab832624bd44a31125df769f7fb"
server_key_iv: "aaf6928791d86ba8509518b3a8bf76a0"
server_key_version: "kuduclusterkey@0"
I20260811 22:15:00.517163 31620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:00.541910 31620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:00.547708 31620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:00.548033 31620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:00.549319 31620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:00.551369 31620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:00.551491 31620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:00.551613 31620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:00.551709 31620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:00.630997 31620 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:45841
I20260811 22:15:00.631011 31888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:45841 every 8 connection(s)
I20260811 22:15:00.632920 31620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:00.635489 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 31620
I20260811 22:15:00.636358 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:38157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:42139
--local_ip_for_outbound_sockets=127.10.173.130
--tserver_master_addrs=127.10.173.190:41969
--webserver_port=37657
--webserver_interface=127.10.173.130
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.173.148:38157/kms with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:15:00 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486500, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:00.707898 31893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:00.636686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41969 (local address 127.10.173.129:37535)
0811 22:15:00.677943 (+ 41257us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:00.678012 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:00.681050 (+ 3038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:00.683689 (+ 2639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:00.683701 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:00.684358 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:00.685134 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:00.685158 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.687985 (+ 2827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.687991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:00.689999 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:00.690030 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.691758 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.691769 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:00.692051 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.692964 (+ 913us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:00.692994 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:00.696033 (+ 3039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:00.702102 (+ 6069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:00.702112 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:00.702126 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:00.702940 (+ 814us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:00.703496 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:00.703501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:00.703504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:00.703760 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:00.705151 (+ 1391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:00.705159 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:00.705585 (+ 426us) client_negotiation.cc:769] Sending connection context
0811 22:15:00.705923 (+ 338us) client_negotiation.cc:240] Negotiation successful
0811 22:15:00.706574 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40842,"thread_start_us":41520,"threads_started":1}
I20260811 22:15:00.709985 31894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:00.637929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:37535 (local address 127.10.173.190:41969)
0811 22:15:00.676898 (+ 38969us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:00.676927 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:00.678241 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 22:15:00.681952 (+ 3711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:00.681971 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:00.682553 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:00.682969 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:00.685398 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.687785 (+ 2387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.690219 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:00.691469 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:00.691704 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:00.696372 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:00.696384 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:00.696401 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:00.696453 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:00.701911 (+ 5458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:00.703119 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:00.703128 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:00.703135 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:00.703345 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:00.704143 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:00.704148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:00.704151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:00.704830 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:00.705045 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:00.706823 (+ 1778us) server_negotiation.cc:299] Negotiation successful
0811 22:15:00.707296 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38552,"thread_start_us":38926,"threads_started":1}
I20260811 22:15:00.713613 31890 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41969
I20260811 22:15:00.713980 31890 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:00.714948 31890 heartbeater.cc:511] Master 127.10.173.190:41969 requested a full tablet report, sending...
W20260811 22:15:00.725203 31890 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:41969 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:15:00.725515 31890 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:00.725900 31890 heartbeater.cc:511] Master 127.10.173.190:41969 requested a full tablet report, sending...
I20260811 22:15:00.727563 31890 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:00.727929 31890 heartbeater.cc:511] Master 127.10.173.190:41969 requested a full tablet report, sending...
W20260811 22:15:00.729347 31890 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:15:01.122006 31895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:01.122466 31895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:01.122716 31895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:01.134656 31895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:01.134842 31895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:01.135077 31895 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:15:01.152066 31895 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.173.148:38157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:42139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=37657
--tserver_master_addrs=127.10.173.190:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:01.154111 31895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:01.156028 31895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:01.173874 31919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:01.176858 31920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:01.194741 31895 server_base.cc:1034] running on GCE node
W20260811 22:15:01.195932 31926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:01.197085 31895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:01.210052 31895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:01.213577 31895 hybrid_clock.cc:648] HybridClock initialized: now 1786486501213463 us; error 97 us; skew 500 ppm
I20260811 22:15:01.214098 31895 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:01 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486501, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:01.221174 31895 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:15:01.223351 31895 webserver.cc:460] Webserver started at http://127.10.173.130:37657/ using document root <none> and password file <none>
I20260811 22:15:01.224264 31895 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:01.224429 31895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:15:01 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486501, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:15:01.249859 31895 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:15:01.253290 31938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:01.254940 31895 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:01.255156 31895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "38566f6eb374433e92ceb023e116438d"
format_stamp: "Formatted at 2026-08-11 22:14:57 on dist-test-slave-68xk"
server_key: "9031440ab9cf259678c790f3945d5c43"
server_key_iv: "6cfe95460adbc4cb45c0a726d9aa9aca"
server_key_version: "kuduclusterkey@0"
I20260811 22:15:01.256279 31895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 11 22:15:01 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486501, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:01.285837 31895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:01.290977 31895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:01.291385 31895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:01.301275 31895 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:01.306854 31895 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:01.307032 31895 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:01.307194 31895 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:01.307291 31895 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:01.373957 31895 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:42139
I20260811 22:15:01.374049 32129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:42139 every 8 connection(s)
I20260811 22:15:01.376006 31895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:01.380975 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 31895
I20260811 22:15:01.381996 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.173.148:38157/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:44253
--local_ip_for_outbound_sockets=127.10.173.131
--tserver_master_addrs=127.10.173.190:41969
--webserver_port=44085
--webserver_interface=127.10.173.131
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:33859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.173.148:38157/kms with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:15:01 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486501, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:01.429761 32134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:01.380172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:56707 (local address 127.10.173.190:41969)
0811 22:15:01.402951 (+ 22779us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:01.402959 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:01.406748 (+ 3789us) server_negotiation.cc:407] Connection header received
0811 22:15:01.406828 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:01.406834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:01.406937 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:01.407040 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:01.409354 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:01.410604 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:01.413542 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:01.414221 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:01.414322 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:01.421773 (+ 7451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:01.421782 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:01.421788 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:01.421837 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:01.425609 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:01.426723 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:01.426730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:01.426734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:01.426884 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:01.427464 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:01.427469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:01.427473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:01.427863 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:01.427989 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:01.429099 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0811 22:15:01.429344 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22617,"thread_start_us":22509,"threads_started":1}
I20260811 22:15:01.432406 32133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:01.379601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41969 (local address 127.10.173.130:56707)
0811 22:15:01.403637 (+ 24036us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:01.403699 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:01.406301 (+ 2602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:01.407510 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:01.407523 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:01.408220 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:01.409111 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:01.409151 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:01.410846 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:01.410852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:01.413354 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:01.413371 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:01.414718 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:01.414726 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:01.415033 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:01.418620 (+ 3587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:01.418655 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:01.421436 (+ 2781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:01.425863 (+ 4427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:01.425873 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:01.425895 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:01.426563 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:01.427040 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:01.427045 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:01.427049 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:01.427326 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:01.428094 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:01.428103 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:01.428580 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 22:15:01.430326 (+ 1746us) client_negotiation.cc:240] Negotiation successful
0811 22:15:01.430958 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23657,"thread_start_us":23429,"threads_started":1}
I20260811 22:15:01.434387 32130 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41969
I20260811 22:15:01.434779 32130 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:01.435771 32130 heartbeater.cc:511] Master 127.10.173.190:41969 requested a full tablet report, sending...
I20260811 22:15:01.438045 31564 ts_manager.cc:194] Registered new tserver with Master: 38566f6eb374433e92ceb023e116438d (127.10.173.130:42139)
I20260811 22:15:01.440263 31564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:56707
Aug 11 22:15:01 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486501, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:15:01.729470 31890 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:01.729907 31890 heartbeater.cc:511] Master 127.10.173.190:41969 requested a full tablet report, sending...
I20260811 22:15:01.731593 31564 ts_manager.cc:194] Registered new tserver with Master: a8152657189f43b4b1b009c3cc4abd89 (127.10.173.129:45841)
I20260811 22:15:01.732853 31564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:37535
W20260811 22:15:01.848079 32138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:01.848497 32138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:01.848685 32138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:01.860019 32138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:01.860239 32138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:01.860452 32138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:15:01.877161 32138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:33859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.173.148:38157/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:44253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=44085
--tserver_master_addrs=127.10.173.190:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:01.879040 32138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:01.880926 32138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:01.902184 32148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:01.903162 32147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:01.906919 32138 server_base.cc:1034] running on GCE node
W20260811 22:15:01.907088 32150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:01.908000 32138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:01.909368 32138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:01.910627 32138 hybrid_clock.cc:648] HybridClock initialized: now 1786486501910545 us; error 56 us; skew 500 ppm
I20260811 22:15:01.911033 32138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:01 dist-test-slave-68xk krb5kdc[29833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486501, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:01.915688 32138 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:15:01.917742 32138 webserver.cc:460] Webserver started at http://127.10.173.131:44085/ using document root <none> and password file <none>
I20260811 22:15:01.918779 32138 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:01.918942 32138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:15:01 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486501, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:15:01.965729 32138 fs_manager.cc:709] Time spent opening directory manager: real 0.024s user 0.010s sys 0.011s
I20260811 22:15:01.979157 32163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:01.988610 32138 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.008s sys 0.003s
I20260811 22:15:01.988850 32138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "ed3ad898b70542439ec8f7f5763e8c6b"
format_stamp: "Formatted at 2026-08-11 22:14:58 on dist-test-slave-68xk"
server_key: "0a086db175b13bcabf31a278329534ab"
server_key_iv: "172767622759cbe23ff7788563f98adc"
server_key_version: "kuduclusterkey@0"
I20260811 22:15:01.990434 32138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:02.027495 32138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:02.033195 32138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
Aug 11 22:15:02 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486501, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for HTTP/127.10.173.148@KRBTEST.COM
I20260811 22:15:02.033531 32138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:02.046761 32138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:02.048985 32138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:02.049106 32138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:02.049232 32138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:02.049310 32138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:02.281350 32138 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:44253
I20260811 22:15:02.281397 32342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:44253 every 8 connection(s)
I20260811 22:15:02.283344 32138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:02.291815 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 32138
Aug 11 22:15:02 dist-test-slave-68xk krb5kdc[29833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486501, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:02.316320 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.289558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:36967 (local address 127.10.173.190:41969)
0811 22:15:02.290646 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.290653 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.292020 (+ 1367us) server_negotiation.cc:407] Connection header received
0811 22:15:02.295412 (+ 3392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.295422 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.295557 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.295688 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:02.298436 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.299995 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.303177 (+ 3182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.303646 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.303728 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.308186 (+ 4458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:02.308194 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:02.308198 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:02.308242 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:02.312107 (+ 3865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.313164 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.313171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.313175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.313427 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.314039 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.314045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.314048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.314437 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:02.314592 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.315569 (+ 977us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.315814 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":927,"thread_start_us":779,"threads_started":1}
I20260811 22:15:02.317615 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.288825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:41969 (local address 127.10.173.131:36967)
0811 22:15:02.291249 (+ 2424us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:02.291312 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:02.294858 (+ 3546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:02.296273 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:02.296290 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:02.297117 (+ 827us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:02.298159 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.298191 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.300281 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.300289 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.302977 (+ 2688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.302993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.304059 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.304063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.304385 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.305301 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:02.305340 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:02.307926 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:02.312283 (+ 4357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:02.312295 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:02.312318 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:02.313001 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:02.313573 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:02.313579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:02.313582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:02.313873 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:02.314585 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:02.314595 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:02.315083 (+ 488us) client_negotiation.cc:769] Sending connection context
0811 22:15:02.315452 (+ 369us) client_negotiation.cc:240] Negotiation successful
0811 22:15:02.316050 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2111,"thread_start_us":1819,"threads_started":1}
I20260811 22:15:02.319372 32345 heartbeater.cc:347] Connected to a master server at 127.10.173.190:41969
I20260811 22:15:02.319736 32345 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:02.320667 32345 heartbeater.cc:511] Master 127.10.173.190:41969 requested a full tablet report, sending...
I20260811 22:15:02.321501 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.292874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.10.173.190:41969)
0811 22:15:02.302920 (+ 10046us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.302926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.303891 (+ 965us) server_negotiation.cc:407] Connection header received
0811 22:15:02.303955 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.303960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.304059 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.304188 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:02.306440 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.307681 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.309444 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.309959 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.310143 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.313311 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:02.313318 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:02.313321 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:02.313353 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:02.318395 (+ 5042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.319274 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.319280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.319283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.319441 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.319919 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.319924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.319928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.320204 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:02.320491 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.320882 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.321110 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9888,"thread_start_us":9746,"threads_started":1}
I20260811 22:15:02.322571 31563 ts_manager.cc:194] Registered new tserver with Master: ed3ad898b70542439ec8f7f5763e8c6b (127.10.173.131:44253)
I20260811 22:15:02.323956 31563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:36967
I20260811 22:15:02.324378 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:15:02.342254 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.328553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48778 (local address 127.10.173.190:41969)
0811 22:15:02.328779 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.328787 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.329048 (+ 261us) server_negotiation.cc:407] Connection header received
0811 22:15:02.329271 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.329276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.329367 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.329476 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:02.331119 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.331841 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.333265 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.333697 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.333835 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.336685 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:02.336700 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:02.336711 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:02.336747 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:02.339507 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.340206 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.340212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.340216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.340363 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.340801 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.340806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.340810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.341077 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:02.341343 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.341668 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.341931 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:15:02.348996 31563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:15:02.357057 31563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:02.401206 32396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.373040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:42139 (local address 127.0.0.1:47088)
0811 22:15:02.390832 (+ 17792us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:02.390863 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:02.391021 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:02.392457 (+ 1436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:02.392496 (+ 39us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:02.392534 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:02.393900 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.393913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.395930 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.395940 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.398365 (+ 2425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.398378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.400132 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.400138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.400317 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.400357 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:15:02.400475 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:15:02.400671 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17533,"mutex_wait_us":375,"thread_start_us":17794,"threads_started":1}
I20260811 22:15:02.402082 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.373224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43814 (local address 127.10.173.131:44253)
0811 22:15:02.391951 (+ 18727us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.391960 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.391997 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:15:02.392094 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.392100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.392325 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.392538 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:02.393884 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.395272 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.399093 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.400550 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.400746 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.400911 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.401449 (+ 538us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.401623 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18525,"thread_start_us":18725,"threads_started":1}
I20260811 22:15:02.402279 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.373697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.10.173.130:42139)
0811 22:15:02.391633 (+ 17936us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.391642 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.391670 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:15:02.391769 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.391776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.391983 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.392158 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:02.394085 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.395725 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.398558 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.399910 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.400115 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.400296 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.400549 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.400751 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17738,"thread_start_us":17588,"threads_started":1}
I20260811 22:15:02.402948 32390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.372840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:44253 (local address 127.0.0.1:43814)
0811 22:15:02.387844 (+ 15004us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:02.387922 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:02.388191 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:02.392537 (+ 4346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:02.392542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:02.392566 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:02.393596 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.393616 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.395440 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.395446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.398914 (+ 3468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.398924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.400801 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.400810 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.401011 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.401048 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:15:02.401161 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:15:02.402373 (+ 1212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14604,"mutex_wait_us":629,"spinlock_wait_cycles":7552,"thread_start_us":15626,"threads_started":1}
I20260811 22:15:02.403779 32392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.373699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45841 (local address 127.0.0.1:55348)
0811 22:15:02.391133 (+ 17434us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:02.391153 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:02.391306 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:02.392256 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:02.392264 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:02.392311 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:02.393359 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.393376 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.395642 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.395648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.398906 (+ 3258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.398924 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.402788 (+ 3864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.402796 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.403259 (+ 463us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.403301 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 22:15:02.403404 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 22:15:02.403503 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17295,"mutex_wait_us":667,"thread_start_us":16706,"threads_started":1}
I20260811 22:15:02.404513 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.374444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55348 (local address 127.10.173.129:45841)
0811 22:15:02.390152 (+ 15708us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.390167 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.391458 (+ 1291us) server_negotiation.cc:407] Connection header received
0811 22:15:02.391593 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.391604 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.391903 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.392149 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:02.393618 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.395476 (+ 1858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.401015 (+ 5539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.402546 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.402754 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.402900 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.404125 (+ 1225us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.404272 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15484,"thread_start_us":15335,"threads_started":1}
I20260811 22:15:02.406649 32024 tablet_service.cc:1467] Processing CreateTablet for tablet 566b016613834e3994ac650f5930afb1 (DEFAULT_TABLE table=test-table [id=5f6004f1361c48b9b81dbe4abbb865d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:02.406649 32241 tablet_service.cc:1467] Processing CreateTablet for tablet 566b016613834e3994ac650f5930afb1 (DEFAULT_TABLE table=test-table [id=5f6004f1361c48b9b81dbe4abbb865d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:02.408205 32024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 566b016613834e3994ac650f5930afb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:02.408205 32241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 566b016613834e3994ac650f5930afb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:02.408119 31773 tablet_service.cc:1467] Processing CreateTablet for tablet 566b016613834e3994ac650f5930afb1 (DEFAULT_TABLE table=test-table [id=5f6004f1361c48b9b81dbe4abbb865d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:02.409389 31773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 566b016613834e3994ac650f5930afb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:02.423588 32398 tablet_bootstrap.cc:492] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: Bootstrap starting.
I20260811 22:15:02.424723 32400 tablet_bootstrap.cc:492] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: Bootstrap starting.
I20260811 22:15:02.426789 32403 tablet_bootstrap.cc:492] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: Bootstrap starting.
I20260811 22:15:02.430051 32398 tablet_bootstrap.cc:654] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:02.430796 32403 tablet_bootstrap.cc:654] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:02.431679 32398 log.cc:824] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:02.432235 32403 log.cc:824] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:02.432991 32400 tablet_bootstrap.cc:654] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:02.434218 32400 log.cc:824] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:02.435539 32398 tablet_bootstrap.cc:492] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: No bootstrap required, opened a new log
I20260811 22:15:02.435604 32403 tablet_bootstrap.cc:492] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: No bootstrap required, opened a new log
I20260811 22:15:02.435895 32398 ts_tablet_manager.cc:1397] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260811 22:15:02.436128 32403 ts_tablet_manager.cc:1397] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 22:15:02.437872 32400 tablet_bootstrap.cc:492] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: No bootstrap required, opened a new log
I20260811 22:15:02.438227 32400 ts_tablet_manager.cc:1397] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260811 22:15:02.444483 32398 raft_consensus.cc:348] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 44253 } } peers { permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 } } peers { permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 } }
I20260811 22:15:02.444964 32398 raft_consensus.cc:374] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:02.445020 32130 heartbeater.cc:503] Master 127.10.173.190:41969 was elected leader, sending a full tablet report...
I20260811 22:15:02.445106 32398 raft_consensus.cc:729] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8152657189f43b4b1b009c3cc4abd89, State: Initialized, Role: FOLLOWER
I20260811 22:15:02.445801 32398 consensus_queue.cc:260] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 44253 } } peers { permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 } } peers { permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 } }
I20260811 22:15:02.447981 31890 heartbeater.cc:503] Master 127.10.173.190:41969 was elected leader, sending a full tablet report...
I20260811 22:15:02.448115 32403 raft_consensus.cc:348] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 44253 } } peers { permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 } } peers { permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 } }
I20260811 22:15:02.448596 32403 raft_consensus.cc:374] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:02.448715 32403 raft_consensus.cc:729] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38566f6eb374433e92ceb023e116438d, State: Initialized, Role: FOLLOWER
I20260811 22:15:02.448777 32398 ts_tablet_manager.cc:1428] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 22:15:02.449404 32403 consensus_queue.cc:260] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 44253 } } peers { permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 } } peers { permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 } }
I20260811 22:15:02.452600 32403 ts_tablet_manager.cc:1428] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: Time spent starting tablet: real 0.016s user 0.010s sys 0.007s
I20260811 22:15:02.454038 32400 raft_consensus.cc:348] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 44253 } } peers { permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 } } peers { permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 } }
I20260811 22:15:02.454492 32400 raft_consensus.cc:374] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:02.454617 32400 raft_consensus.cc:729] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed3ad898b70542439ec8f7f5763e8c6b, State: Initialized, Role: FOLLOWER
I20260811 22:15:02.455212 32400 consensus_queue.cc:260] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 44253 } } peers { permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 } } peers { permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 } }
I20260811 22:15:02.456423 32345 heartbeater.cc:503] Master 127.10.173.190:41969 was elected leader, sending a full tablet report...
I20260811 22:15:02.457051 32400 ts_tablet_manager.cc:1428] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: Time spent starting tablet: real 0.019s user 0.009s sys 0.008s
I20260811 22:15:02.513324 32414 raft_consensus.cc:482] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:02.513684 32414 raft_consensus.cc:504] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 44253 } } peers { permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 } } peers { permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 } }
I20260811 22:15:02.515525 32414 leader_election.cc:288] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38566f6eb374433e92ceb023e116438d (127.10.173.130:42139), a8152657189f43b4b1b009c3cc4abd89 (127.10.173.129:45841)
I20260811 22:15:02.522404 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.516067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:39245 (local address 127.10.173.130:42139)
0811 22:15:02.516287 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.516293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.516317 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:15:02.516565 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.516570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.516653 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.516752 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:02.517298 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.518337 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.520485 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.521376 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.521508 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.521561 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.521978 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.522108 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:15:02.522471 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.515835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:42139 (local address 127.10.173.131:39245)
0811 22:15:02.516173 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:02.516198 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:02.516372 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:02.516801 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:02.516805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:02.516832 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:02.517129 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.517143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.518517 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.518523 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.520309 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.520326 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.521566 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.521573 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.521795 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.521834 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:15:02.521936 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:15:02.522087 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":27}
I20260811 22:15:02.522877 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.516079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:38455 (local address 127.10.173.129:45841)
0811 22:15:02.516288 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.516294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.516927 (+ 633us) server_negotiation.cc:407] Connection header received
0811 22:15:02.517065 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.517070 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.517157 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.517270 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:02.517871 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.519017 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.520830 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.521827 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.522000 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.522073 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.522448 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.522600 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:15:02.523264 32417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.515944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45841 (local address 127.10.173.131:38455)
0811 22:15:02.516783 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:02.516802 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:02.516943 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:02.517299 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:02.517304 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:02.517332 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:02.517702 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.517714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.519170 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.519175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.520659 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:02.520674 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.522045 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.522051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:02.522243 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.522278 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:15:02.522373 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:15:02.522909 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":22,"thread_start_us":479,"threads_started":1}
I20260811 22:15:02.523711 32057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566b016613834e3994ac650f5930afb1" candidate_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38566f6eb374433e92ceb023e116438d" is_pre_election: true
I20260811 22:15:02.524405 32057 raft_consensus.cc:2454] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed3ad898b70542439ec8f7f5763e8c6b in term 0.
I20260811 22:15:02.524405 31805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566b016613834e3994ac650f5930afb1" candidate_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8152657189f43b4b1b009c3cc4abd89" is_pre_election: true
I20260811 22:15:02.525022 31805 raft_consensus.cc:2454] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed3ad898b70542439ec8f7f5763e8c6b in term 0.
I20260811 22:15:02.525242 32170 leader_election.cc:302] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38566f6eb374433e92ceb023e116438d, ed3ad898b70542439ec8f7f5763e8c6b; no voters:
I20260811 22:15:02.526013 32414 raft_consensus.cc:2781] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:02.526161 32414 raft_consensus.cc:482] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:02.526284 32414 raft_consensus.cc:3037] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:02.528729 32414 raft_consensus.cc:504] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 44253 } } peers { permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 } } peers { permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 } }
I20260811 22:15:02.529369 32414 leader_election.cc:288] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [CANDIDATE]: Term 1 election: Requested vote from peers 38566f6eb374433e92ceb023e116438d (127.10.173.130:42139), a8152657189f43b4b1b009c3cc4abd89 (127.10.173.129:45841)
I20260811 22:15:02.529856 32057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566b016613834e3994ac650f5930afb1" candidate_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38566f6eb374433e92ceb023e116438d"
I20260811 22:15:02.530071 32057 raft_consensus.cc:3037] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:02.529995 31805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "566b016613834e3994ac650f5930afb1" candidate_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8152657189f43b4b1b009c3cc4abd89"
I20260811 22:15:02.530280 31805 raft_consensus.cc:3037] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:02.532318 32057 raft_consensus.cc:2454] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed3ad898b70542439ec8f7f5763e8c6b in term 1.
I20260811 22:15:02.532737 32170 leader_election.cc:302] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38566f6eb374433e92ceb023e116438d, ed3ad898b70542439ec8f7f5763e8c6b; no voters:
I20260811 22:15:02.532801 31805 raft_consensus.cc:2454] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed3ad898b70542439ec8f7f5763e8c6b in term 1.
I20260811 22:15:02.533037 32414 raft_consensus.cc:2781] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:02.533579 32414 raft_consensus.cc:686] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 1 LEADER]: Becoming Leader. State: Replica: ed3ad898b70542439ec8f7f5763e8c6b, State: Running, Role: LEADER
I20260811 22:15:02.534073 32414 consensus_queue.cc:237] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 44253 } } peers { permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 } } peers { permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 } }
I20260811 22:15:02.539343 31560 catalog_manager.cc:5495] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b reported cstate change: term changed from 0 to 1, leader changed from <none> to ed3ad898b70542439ec8f7f5763e8c6b (127.10.173.131). New cstate: current_term: 1 leader_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3ad898b70542439ec8f7f5763e8c6b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 44253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 } health_report { overall_health: UNKNOWN } } }
W20260811 22:15:02.541327 32346 tablet.cc:2369] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:02.557562 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.551387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.10.173.131:44253)
0811 22:15:02.551617 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.551625 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.551763 (+ 138us) server_negotiation.cc:407] Connection header received
0811 22:15:02.552050 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.552054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.552189 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.552360 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:02.552786 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.553591 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.555217 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.555661 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.555814 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.556387 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:02.556921 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:02.557099 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.557177 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.557269 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:15:02.566951 32057 raft_consensus.cc:1261] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d [term 1 FOLLOWER]: Refusing update from remote peer ed3ad898b70542439ec8f7f5763e8c6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:02.566948 31805 raft_consensus.cc:1261] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89 [term 1 FOLLOWER]: Refusing update from remote peer ed3ad898b70542439ec8f7f5763e8c6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:02.568156 32418 consensus_queue.cc:1035] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8152657189f43b4b1b009c3cc4abd89" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:02.568647 32414 consensus_queue.cc:1035] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "38566f6eb374433e92ceb023e116438d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 42139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:02.596789 31563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:02.597064 31563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:02.600803 31563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P db073fe734684b88af8a86180428d125: Sending DeleteTablet for 3 replicas of tablet 566b016613834e3994ac650f5930afb1
I20260811 22:15:02.601995 32024 tablet_service.cc:1514] Processing DeleteTablet for tablet 566b016613834e3994ac650f5930afb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:02 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:47088
I20260811 22:15:02.601995 32241 tablet_service.cc:1514] Processing DeleteTablet for tablet 566b016613834e3994ac650f5930afb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:02 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:43814
I20260811 22:15:02.602110 31773 tablet_service.cc:1514] Processing DeleteTablet for tablet 566b016613834e3994ac650f5930afb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:02 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:55348
I20260811 22:15:02.612109 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 31620
I20260811 22:15:02.615386 32443 tablet_replica.cc:330] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: stopping tablet replica
I20260811 22:15:02.615882 32443 raft_consensus.cc:2229] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:02.616456 32443 raft_consensus.cc:2258] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:02.630091 32440 tablet_replica.cc:330] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: stopping tablet replica
I20260811 22:15:02.630586 32443 ts_tablet_manager.cc:1905] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:02.630875 32440 raft_consensus.cc:2229] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:02.631548 32440 raft_consensus.cc:2258] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:02.632650 32439 tablet_replica.cc:330] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: stopping tablet replica
I20260811 22:15:02.633060 32439 raft_consensus.cc:2229] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:02.633617 32439 raft_consensus.cc:2258] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:02.635756 32439 ts_tablet_manager.cc:1905] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:02.637305 32440 ts_tablet_manager.cc:1905] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:02.638656 32443 ts_tablet_manager.cc:1918] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:02.638849 32443 log.cc:1196] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/566b016613834e3994ac650f5930afb1
I20260811 22:15:02.639456 32443 ts_tablet_manager.cc:1939] T 566b016613834e3994ac650f5930afb1 P a8152657189f43b4b1b009c3cc4abd89: Deleting consensus metadata
I20260811 22:15:02.640000 32397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.614677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.10.173.129:45841)
0811 22:15:02.614925 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.614939 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.614964 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:15:02.615055 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.615063 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.615187 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.615326 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:02.619002 (+ 3676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.620499 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.625573 (+ 5074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.626538 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.626702 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.630991 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:02.631052 (+ 61us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:02.631070 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:02.631121 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:02.635582 (+ 4461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.636628 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.636639 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.636645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.636827 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.637321 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.637329 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.637335 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.637901 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:02.638245 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.639094 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.639394 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:15:02.641453 31883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:02.641455 31536 catalog_manager.cc:4841] TS a8152657189f43b4b1b009c3cc4abd89 (127.10.173.129:45841): tablet 566b016613834e3994ac650f5930afb1 (table test-table [id=5f6004f1361c48b9b81dbe4abbb865d3]) successfully deleted
I20260811 22:15:02.643049 32439 ts_tablet_manager.cc:1918] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:02.643241 32439 log.cc:1196] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/566b016613834e3994ac650f5930afb1
I20260811 22:15:02.643769 32439 ts_tablet_manager.cc:1939] T 566b016613834e3994ac650f5930afb1 P 38566f6eb374433e92ceb023e116438d: Deleting consensus metadata
I20260811 22:15:02.645581 31542 catalog_manager.cc:4841] TS 38566f6eb374433e92ceb023e116438d (127.10.173.130:42139): tablet 566b016613834e3994ac650f5930afb1 (table test-table [id=5f6004f1361c48b9b81dbe4abbb865d3]) successfully deleted
I20260811 22:15:02.645763 32440 ts_tablet_manager.cc:1918] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:02.645927 32440 log.cc:1196] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/566b016613834e3994ac650f5930afb1
I20260811 22:15:02.646400 32440 ts_tablet_manager.cc:1939] T 566b016613834e3994ac650f5930afb1 P ed3ad898b70542439ec8f7f5763e8c6b: Deleting consensus metadata
I20260811 22:15:02.649235 31537 catalog_manager.cc:4841] TS ed3ad898b70542439ec8f7f5763e8c6b (127.10.173.131:44253): tablet 566b016613834e3994ac650f5930afb1 (table test-table [id=5f6004f1361c48b9b81dbe4abbb865d3]) successfully deleted
I20260811 22:15:02.738598 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 31620
I20260811 22:15:02.765707 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 31895
I20260811 22:15:02.782043 32395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.766564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.10.173.130:42139)
0811 22:15:02.766765 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.766772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.766795 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:15:02.766983 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.766990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.767079 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.767243 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:02.768718 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.769825 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.771223 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.771933 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.772065 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.774797 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:02.774819 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:02.774841 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:02.774883 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:02.778899 (+ 4016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.779704 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.779715 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.779721 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.779893 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.780299 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.780306 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.780310 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.780773 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:02.781110 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.781391 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.781626 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:15:02.782971 32123 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:02.886165 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 31895
I20260811 22:15:02.912695 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 32138
I20260811 22:15:02.928810 32394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:02.913778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43832 (local address 127.10.173.131:44253)
0811 22:15:02.913953 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:02.913958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:02.913977 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:15:02.914059 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:02.914062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:02.914140 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:02.914224 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:02.916223 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.917072 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.918758 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:02.919218 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:02.919372 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:02.922081 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:02.922096 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:02.922113 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:02.922175 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:02.925466 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.926315 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.926322 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.926325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.926560 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:02.927018 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:02.927024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:02.927027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:02.927518 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:02.927872 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:02.928183 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 22:15:02.928405 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:15:02.929865 32333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:03.059357 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 32138
I20260811 22:15:03.086884 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 31511
I20260811 22:15:03.088131 31602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:03.204630 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 31511
I20260811 22:15:03.239082 10934 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:15:03.584801 10934 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:15:03.584990 10934 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:15:03.941445 10934 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:15:03.941 UTC [29886] LOG: received smart shutdown request
2026-08-11 22:15:03.945 UTC [29886] LOG: background worker "logical replication launcher" (PID 29899) exited with exit code 1
2026-08-11 22:15:03.945 UTC [29892] LOG: shutting down
2026-08-11 22:15:04.148 UTC [29886] LOG: database system is shut down
2026-08-11T22:15:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55158 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:04 dist-test-slave-68xk krb5kdc[32465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:04 dist-test-slave-68xk krb5kdc[32465](info): set up 2 sockets
Aug 11 22:15:04 dist-test-slave-68xk krb5kdc[32465](info): commencing operation
krb5kdc: starting...
W20260811 22:15:05.639643 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:15:05 dist-test-slave-68xk krb5kdc[32465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:15:05.942804 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:40897
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:40897,127.10.173.189:41363,127.10.173.188:43487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:06.286900 32483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:06.287209 32483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:06.287367 32483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:06.295125 32483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:06.295250 32483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:06.295327 32483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:06.295421 32483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:06.295542 32483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:06.305981 32483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:40897,127.10.173.189:41363,127.10.173.188:43487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:40897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:06.307471 32483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:06.308880 32483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:06.321724 32494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.321918 32490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.321668 32492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:06.322917 32483 server_base.cc:1034] running on GCE node
I20260811 22:15:06.323910 32483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:06.325512 32483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:06.326781 32483 hybrid_clock.cc:648] HybridClock initialized: now 1786486506326719 us; error 52 us; skew 500 ppm
I20260811 22:15:06.327222 32483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:06 dist-test-slave-68xk krb5kdc[32465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:06.336154 32483 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:15:06.337739 32483 webserver.cc:460] Webserver started at http://127.10.173.190:43947/ using document root <none> and password file <none>
I20260811 22:15:06.338590 32483 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:06.338716 32483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:06.339026 32483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:06.342499 32483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-68xk"
server_key: "92d02189cded5e22fb1bc5fa5ac0272a"
server_key_iv: "40953bbf4416ee74cf36c56c7de6df98"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.343405 32483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-68xk"
server_key: "92d02189cded5e22fb1bc5fa5ac0272a"
server_key_iv: "40953bbf4416ee74cf36c56c7de6df98"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.349218 32483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 22:15:06.353015 32504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.354604 32483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:15:06.354790 32483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-68xk"
server_key: "92d02189cded5e22fb1bc5fa5ac0272a"
server_key_iv: "40953bbf4416ee74cf36c56c7de6df98"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.354976 32483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:06.371776 32483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:06.376259 32483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:06.376605 32483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:06.397920 32483 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:40897
I20260811 22:15:06.397951 32598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:40897 every 8 connection(s)
I20260811 22:15:06.399524 32483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:06.400941 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 32483
I20260811 22:15:06.401213 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:15:06.402086 10934 external_mini_cluster.cc:1428] Setting key b8fa0ba3e7c77408d131efd070ea0d00
I20260811 22:15:06.404428 32599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:06.414738 32599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.173.189@KRBTEST.COM; defaulting to no policy
Aug 11 22:15:06 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.189@KRBTEST.COM" created.
I20260811 22:15:06.440719 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.418273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:40897 (local address 127.0.0.1:56772)
0811 22:15:06.419486 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.419554 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.421718 (+ 2164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.422837 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.422842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.423428 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.423784 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.423802 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.426258 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.426262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.427539 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.427551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.429026 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.429032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.429158 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.429892 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.429926 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.432354 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.436605 (+ 4251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.436615 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.436624 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.437027 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.437534 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.437538 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.437540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.437714 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.438805 (+ 1091us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.438811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.439014 (+ 203us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.439229 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.439625 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":545,"threads_started":1}
I20260811 22:15:06.440819 32603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.418266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56772 (local address 127.10.173.190:40897)
0811 22:15:06.420179 (+ 1913us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.420193 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.420250 (+ 57us) server_negotiation.cc:407] Connection header received
0811 22:15:06.422133 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.422142 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.422551 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.422778 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.424006 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.426084 (+ 2078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.427701 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.428760 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.428992 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.432672 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.432687 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.432707 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.432757 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.436458 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.437183 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.437195 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.437202 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.437414 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.437837 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.437844 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.437850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.438654 (+ 804us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.438861 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.439438 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.439845 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":1686,"spinlock_wait_cycles":33024,"thread_start_us":1470,"threads_started":1}
I20260811 22:15:06.443976 32599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:15:06.445003 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.444621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41610)
0811 22:15:06.444808 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.444903 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 22:15:06.445286 32510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
I20260811 22:15:06.446990 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.446607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41626)
0811 22:15:06.446763 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.446827 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 22:15:06.447212 32599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:41363: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab.
Entry for principal kudu/127.10.173.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab.
I20260811 22:15:06.470242 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.189:41363
--webserver_interface=127.10.173.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab
--principal=kudu/127.10.173.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:40897,127.10.173.189:41363,127.10.173.188:43487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:15:06.496692 32599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } attempt: 1
I20260811 22:15:06.498265 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.497550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41630)
0811 22:15:06.497835 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.498007 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 22:15:06.499493 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.499037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41636)
0811 22:15:06.499225 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.499320 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:15:06.499766 32599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:41363: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
I20260811 22:15:06.568100 32599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } attempt: 2
I20260811 22:15:06.569387 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.568862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41644)
0811 22:15:06.569147 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.569243 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 22:15:06.570631 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.570129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41654)
0811 22:15:06.570327 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.570480 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 22:15:06.570880 32599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:41363: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
I20260811 22:15:06.662135 32599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } attempt: 3
I20260811 22:15:06.663494 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.663002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41664)
0811 22:15:06.663251 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.663348 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 22:15:06.664700 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.664299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41674)
0811 22:15:06.664463 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.664560 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:15:06.664907 32599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:41363: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
I20260811 22:15:06.808211 32599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } attempt: 4
I20260811 22:15:06.809557 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.808991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41690)
0811 22:15:06.809320 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.809417 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260811 22:15:06.810670 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.810244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41698)
0811 22:15:06.810402 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.810547 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:15:06.810858 32599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:41363: Network error: Client connection negotiation failed: client connection to 127.10.173.189:41363: connect: Connection refused (error 111)
W20260811 22:15:06.816215 32608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:06.816529 32608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:06.816685 32608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:06.823877 32608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:06.824011 32608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:06.824115 32608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:06.824221 32608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:06.824296 32608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:06.834033 32608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:40897,127.10.173.189:41363,127.10.173.188:43487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab
--principal=kudu/127.10.173.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.189:41363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.173.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:06.835391 32608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:06.836733 32608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:06.848918 32618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.848866 32623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:06.849007 32619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:06.849813 32608 server_base.cc:1034] running on GCE node
I20260811 22:15:06.850725 32608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:06.852267 32608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:06.853539 32608 hybrid_clock.cc:648] HybridClock initialized: now 1786486506853492 us; error 56 us; skew 500 ppm
I20260811 22:15:06.853945 32608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:06 dist-test-slave-68xk krb5kdc[32465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:06.857517 32608 init.cc:377] Logged in from keytab as kudu/127.10.173.189@KRBTEST.COM (short username kudu)
I20260811 22:15:06.859262 32608 webserver.cc:460] Webserver started at http://127.10.173.189:35583/ using document root <none> and password file <none>
I20260811 22:15:06.860013 32608 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:06.860291 32608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:06.860663 32608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:06.864019 32608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/data/instance:
uuid: "03c2e84d56534fda8c59ed81e638092e"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-68xk"
server_key: "e6a5ad9c08964680186a893b0e097b4e"
server_key_iv: "a5536711517dd8bd582803c302d0f424"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.864940 32608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/wal/instance:
uuid: "03c2e84d56534fda8c59ed81e638092e"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-68xk"
server_key: "e6a5ad9c08964680186a893b0e097b4e"
server_key_iv: "a5536711517dd8bd582803c302d0f424"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.870519 32608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 22:15:06.874236 32632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:06.875594 32608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:06.875792 32608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/wal
uuid: "03c2e84d56534fda8c59ed81e638092e"
format_stamp: "Formatted at 2026-08-11 22:15:06 on dist-test-slave-68xk"
server_key: "e6a5ad9c08964680186a893b0e097b4e"
server_key_iv: "a5536711517dd8bd582803c302d0f424"
server_key_version: "encryptionkey@0"
I20260811 22:15:06.875973 32608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:06.897913 32608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:06.902127 32608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:06.902487 32608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:06.921660 32608 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.189:41363
I20260811 22:15:06.921697 32716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.189:41363 every 8 connection(s)
I20260811 22:15:06.923182 32608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
I20260811 22:15:06.927170 32718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:06.929338 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 32608
I20260811 22:15:06.929562 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-1/wal/instance
I20260811 22:15:06.930024 10934 external_mini_cluster.cc:1428] Setting key cc8f87b622bc6caa3240a31124235164
I20260811 22:15:06.933854 32718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:15:06 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:06.956519 32603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.936638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56776 (local address 127.10.173.190:40897)
0811 22:15:06.936847 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.936852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.938343 (+ 1491us) server_negotiation.cc:407] Connection header received
0811 22:15:06.940337 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.940343 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.940438 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.940553 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.942045 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.942799 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.945303 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.945776 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.945899 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.950278 (+ 4379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.950283 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.950286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.950321 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.953211 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.953978 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.953986 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.953991 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.954121 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.954643 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.954649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.954652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.955015 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.955202 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.955946 (+ 744us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.956197 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:15:06.957538 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.937331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:40897 (local address 127.0.0.1:56776)
0811 22:15:06.938161 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.938206 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.940039 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.940801 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.940809 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.941332 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.941871 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.941887 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.942965 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.942973 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.945119 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.945137 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.946032 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.946041 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.946370 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.947164 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.947220 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.949981 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.953367 (+ 3386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.953376 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.953392 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.953848 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.954258 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.954263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.954267 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.954528 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.955162 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.955170 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.955543 (+ 373us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.955829 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.956340 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":379,"threads_started":1}
WARNING: no policy specified for kudu/127.10.173.188@KRBTEST.COM; defaulting to no policy
I20260811 22:15:06.959542 32718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.188@KRBTEST.COM" created.
Aug 11 22:15:06 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
I20260811 22:15:06.974107 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.960314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41706)
0811 22:15:06.960523 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.960538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:06.960627 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:06.961959 (+ 1332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:06.961964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:06.962392 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:06.962578 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.962586 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.963907 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.963913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.964752 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:06.964763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.965595 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.965600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:06.965863 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.966542 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:06.966559 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:06.968536 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:06.971649 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.971657 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.971661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.972013 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.972408 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:06.972414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:06.972417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:06.972569 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:06.973125 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:06.973131 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:06.973270 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 22:15:06.973421 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 22:15:06.973698 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":44}
I20260811 22:15:06.974205 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.960755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.10.173.189:41363)
0811 22:15:06.961504 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:06.961514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:06.961548 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:15:06.961605 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:06.961621 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:06.961805 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:06.961964 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:06.962729 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.963774 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.964899 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:06.965448 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:06.965587 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:06.968748 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:06.968758 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:06.968771 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:06.968802 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:06.971515 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.972143 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.972158 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.972161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.972299 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:06.972672 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:06.972679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:06.972682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:06.973003 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:06.973233 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:06.973559 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 22:15:06.973798 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":553,"thread_start_us":352,"threads_started":1}
I20260811 22:15:06.976805 32718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:15:06.978155 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.977568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50234)
0811 22:15:06.977803 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.977946 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 22:15:06.978583 32636 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
I20260811 22:15:06.980311 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:06.979953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50242)
0811 22:15:06.980133 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:06.980210 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:15:06.980665 32718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:43487: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab.
Entry for principal kudu/127.10.173.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab.
I20260811 22:15:06.994670 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.188:43487
--webserver_interface=127.10.173.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab
--principal=kudu/127.10.173.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:40897,127.10.173.189:41363,127.10.173.188:43487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:15:07.030009 32718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } attempt: 1
I20260811 22:15:07.031364 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.030855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50246)
0811 22:15:07.031088 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.031207 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 22:15:07.032488 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.032122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50252)
0811 22:15:07.032308 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.032379 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:15:07.032711 32718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:43487: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
I20260811 22:15:07.100936 32718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } attempt: 2
I20260811 22:15:07.102193 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.101689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50266)
0811 22:15:07.101924 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.102022 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 22:15:07.103206 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.102871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50280)
0811 22:15:07.103015 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.103089 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:15:07.103434 32718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:43487: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
I20260811 22:15:07.110093 32599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } attempt: 5
Aug 11 22:15:07 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
I20260811 22:15:07.124941 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.110792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41710)
0811 22:15:07.110993 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.111015 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.111131 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.111605 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.111610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.112000 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.112200 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.112210 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.113666 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.113670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.114654 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.114668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.115626 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.115630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.115768 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.116479 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.116503 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.118619 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.122641 (+ 4022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.122645 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.122647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.123023 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.123415 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.123418 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.123420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.123552 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.124153 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.124156 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.124290 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.124407 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.124613 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 22:15:07.125099 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.110898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41710 (local address 127.10.173.189:41363)
0811 22:15:07.111117 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.111123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.111150 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:15:07.111235 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.111242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.111368 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.111549 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.112394 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.113500 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.114806 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.115448 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.115578 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.118844 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.118853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.118859 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.118902 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.122455 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.123149 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.123154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.123158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.123305 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.123665 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.123670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.123673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.124032 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.124185 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.124552 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.124787 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:15:07.126176 32599 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:15:07.127413 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.126990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50288)
0811 22:15:07.127182 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.127279 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 22:15:07.128468 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.128134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50294)
0811 22:15:07.128279 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.128363 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 22:15:07.128780 32599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:43487: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
I20260811 22:15:07.180050 32599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } attempt: 1
I20260811 22:15:07.181457 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.180924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50310)
0811 22:15:07.181165 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.181266 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 22:15:07.182578 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.182159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50326)
0811 22:15:07.182316 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.182404 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 22:15:07.182802 32599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:43487: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
I20260811 22:15:07.194681 32718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } attempt: 3
I20260811 22:15:07.195851 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.195380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50328)
0811 22:15:07.195582 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.195690 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:15:07.196933 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.196507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50334)
0811 22:15:07.196733 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.196820 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260811 22:15:07.197237 32718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:43487: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
I20260811 22:15:07.251045 32599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } attempt: 2
I20260811 22:15:07.252231 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.251746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50338)
0811 22:15:07.251951 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.252066 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:15:07.253263 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.252912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50346)
0811 22:15:07.253070 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.253156 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:15:07.253497 32599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:43487: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
I20260811 22:15:07.340442 32718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } attempt: 4
I20260811 22:15:07.341758 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.341117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50354)
0811 22:15:07.341431 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.341542 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 22:15:07.342917 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.342544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50370)
0811 22:15:07.342701 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.342791 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:15:07.343142 32718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:43487: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
W20260811 22:15:07.347981 32730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:07.348284 32730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:07.348429 32730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:07.355820 32730 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:07.355921 32730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:07.355996 32730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:07.356081 32730 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:07.356144 32730 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:07.359735 32599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } attempt: 3
I20260811 22:15:07.360981 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.360446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50378)
0811 22:15:07.360707 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.360810 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 22:15:07.362159 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.361725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50388)
0811 22:15:07.361966 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.362051 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260811 22:15:07.362398 32599 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:43487: Network error: Client connection negotiation failed: client connection to 127.10.173.188:43487: connect: Connection refused (error 111)
I20260811 22:15:07.366804 32730 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:40897,127.10.173.189:41363,127.10.173.188:43487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab
--principal=kudu/127.10.173.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.188:43487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.173.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:07.368202 32730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:07.369531 32730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:07.381848 32743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.381955 32751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:07.381857 32745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:07.383252 32730 server_base.cc:1034] running on GCE node
I20260811 22:15:07.384039 32730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:07.385516 32730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:07.386767 32730 hybrid_clock.cc:648] HybridClock initialized: now 1786486507386703 us; error 85 us; skew 500 ppm
I20260811 22:15:07.387190 32730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:07 dist-test-slave-68xk krb5kdc[32465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:07.390503 32730 init.cc:377] Logged in from keytab as kudu/127.10.173.188@KRBTEST.COM (short username kudu)
I20260811 22:15:07.391997 32730 webserver.cc:460] Webserver started at http://127.10.173.188:38181/ using document root <none> and password file <none>
I20260811 22:15:07.392707 32730 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:07.392858 32730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:07.393177 32730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:07.396387 32730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/data/instance:
uuid: "e43f05a7de3d48688d8583766f119933"
format_stamp: "Formatted at 2026-08-11 22:15:07 on dist-test-slave-68xk"
server_key: "c22baa3e62f4f011767de6b7f167c670"
server_key_iv: "733e194e2ed5837617d0cb7379fa8d3c"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.397197 32730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/wal/instance:
uuid: "e43f05a7de3d48688d8583766f119933"
format_stamp: "Formatted at 2026-08-11 22:15:07 on dist-test-slave-68xk"
server_key: "c22baa3e62f4f011767de6b7f167c670"
server_key_iv: "733e194e2ed5837617d0cb7379fa8d3c"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.402686 32730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 22:15:07.406301 32764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:07.407742 32730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:07.407931 32730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/wal
uuid: "e43f05a7de3d48688d8583766f119933"
format_stamp: "Formatted at 2026-08-11 22:15:07 on dist-test-slave-68xk"
server_key: "c22baa3e62f4f011767de6b7f167c670"
server_key_iv: "733e194e2ed5837617d0cb7379fa8d3c"
server_key_version: "encryptionkey@0"
I20260811 22:15:07.408130 32730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:07.435763 32730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:07.439993 32730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:07.440320 32730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:07.459893 32730 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.188:43487
I20260811 22:15:07.459978 382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.188:43487 every 8 connection(s)
I20260811 22:15:07.461298 32730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
I20260811 22:15:07.465353 383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:07.467630 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 32730
I20260811 22:15:07.467872 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486285256683-10934-0/minicluster-data/master-2/wal/instance
I20260811 22:15:07.468379 10934 external_mini_cluster.cc:1428] Setting key e801801448deda3b5c57cc9ddb4dec5a
I20260811 22:15:07.476322 383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:15:07 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
Aug 11 22:15:07 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:07.493253 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.470380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50392 (local address 127.10.173.188:43487)
0811 22:15:07.471436 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.471460 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.471521 (+ 61us) server_negotiation.cc:407] Connection header received
0811 22:15:07.473409 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.473447 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.474023 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.474527 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.475957 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.478193 (+ 2236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.480025 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.480937 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.481149 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.485424 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.485432 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.485443 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.485484 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.489041 (+ 3557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.489786 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.489794 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.489811 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.489957 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.490368 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.490374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.490380 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.491028 (+ 648us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.491369 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.491836 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.492257 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":378,"threads_started":1}
I20260811 22:15:07.496502 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.479700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:40897 (local address 127.0.0.1:56786)
0811 22:15:07.480888 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.480932 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.481097 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.481539 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.481546 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.482277 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.482688 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.482705 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.483926 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.483932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.485157 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.485170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.486043 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.486050 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.486235 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.486981 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.486999 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.489517 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.493182 (+ 3665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.493194 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.493207 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.493936 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.494399 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.494405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.494409 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.494617 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.495231 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.495244 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.495473 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.495694 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.496051 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"thread_start_us":725,"threads_started":1}
I20260811 22:15:07.497035 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.479719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56786 (local address 127.10.173.190:40897)
0811 22:15:07.480388 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.480394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.481180 (+ 786us) server_negotiation.cc:407] Connection header received
0811 22:15:07.481271 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.481275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.481344 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.481474 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.482886 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.483754 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.485324 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.485808 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.485990 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.489742 (+ 3752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.489748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.489752 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.489788 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.493012 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.494085 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.494094 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.494098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.494265 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.494748 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.494756 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.494760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.495081 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.495258 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.495853 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.496108 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":305,"threads_started":1}
I20260811 22:15:07.498669 383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:15:07 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
Aug 11 22:15:07 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:07.514910 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.499523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41724 (local address 127.10.173.189:41363)
0811 22:15:07.499725 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.499732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.499854 (+ 122us) server_negotiation.cc:407] Connection header received
0811 22:15:07.499964 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.499969 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.500043 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.500143 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.501039 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.501686 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.503508 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.504136 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.504248 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.508333 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.508340 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.508351 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.508388 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.511591 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.512474 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.512479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.512482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.512744 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.513292 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.513297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.513300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.513688 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.513867 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.514331 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.514592 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:15:07.514914 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.499539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:41363 (local address 127.0.0.1:41724)
0811 22:15:07.499769 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.499785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.499875 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.500191 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.500195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.500670 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.500874 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.500889 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.501844 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.501851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.503302 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.503323 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.504323 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.504332 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.504576 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.505149 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.505181 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.508088 (+ 2907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.511760 (+ 3672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.511766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.511769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.512334 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.512968 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.512971 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.512974 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.513119 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.513819 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.513823 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.513999 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.514238 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.514575 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
I20260811 22:15:07.516248 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.496266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56790 (local address 127.10.173.190:40897)
0811 22:15:07.496909 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.496917 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.496942 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:15:07.497130 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.497135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.497217 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.497340 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.499164 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.500378 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.502129 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.502617 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.502713 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.509186 (+ 6473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.509194 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.509199 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.509246 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.512449 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.513410 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.513414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.513417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.513545 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.514012 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.514017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.514020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.514394 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.514694 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.515458 (+ 764us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.515756 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":313,"threads_started":1}
I20260811 22:15:07.516229 383 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:15:07 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486507, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
Aug 11 22:15:07 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
I20260811 22:15:07.530198 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.517037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.10.173.188:43487)
0811 22:15:07.517218 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.517224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.517410 (+ 186us) server_negotiation.cc:407] Connection header received
0811 22:15:07.517559 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.517565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.517654 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.517760 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.518625 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.519290 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.520441 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.520861 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.521002 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.524470 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.524479 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.524487 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.524529 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.527254 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.527984 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.527992 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.527995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.528140 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.528541 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.528547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.528550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.528888 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.529001 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.529550 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.529933 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":60}
I20260811 22:15:07.530289 391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.517031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50402)
0811 22:15:07.517285 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.517322 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.517438 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.517799 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.517803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.518276 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.518491 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.518502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.519414 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.519422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.520298 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.520310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.521021 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.521026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.521195 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.521774 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.521790 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.524229 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.527394 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.527400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.527404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.527854 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.528268 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.528272 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.528275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.528421 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.529022 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.529028 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.529210 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.529416 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.529782 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165}
I20260811 22:15:07.535246 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.518247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41732 (local address 127.10.173.189:41363)
0811 22:15:07.518479 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.518486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.518697 (+ 211us) server_negotiation.cc:407] Connection header received
0811 22:15:07.518888 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.518894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.518967 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.519073 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.520469 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.521231 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.522733 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.523463 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.523597 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.528717 (+ 5120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.528722 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.528725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.528758 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.532325 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.533054 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.533062 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.533066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.533249 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.533633 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.533638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.533642 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.534022 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.534342 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.534563 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.534840 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:15:07.537151 383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933: Bootstrap starting.
I20260811 22:15:07.539666 32599 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } attempt: 4
I20260811 22:15:07.541002 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:15:07.541792 383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:07.543164 383 log.cc:824] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:07.546653 383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933: No bootstrap required, opened a new log
Aug 11 22:15:07 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
I20260811 22:15:07.558760 383 raft_consensus.cc:348] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } }
I20260811 22:15:07.559296 383 raft_consensus.cc:374] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:07.559504 383 raft_consensus.cc:729] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e43f05a7de3d48688d8583766f119933, State: Initialized, Role: FOLLOWER
I20260811 22:15:07.559923 32602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.540255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50404)
0811 22:15:07.540491 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.540506 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.540594 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.541130 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.541145 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.541735 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.541996 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.542012 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.543572 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.543579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.545165 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.545181 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.546560 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.546566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.546900 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.547727 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.547753 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.551309 (+ 3556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.556526 (+ 5217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.556535 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.556540 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.557196 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.557690 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.557696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.557700 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.557898 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.558750 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.558756 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.558966 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.559159 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.559489 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260811 22:15:07.560374 383 consensus_queue.cc:260] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } }
I20260811 22:15:07.561793 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.540299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50404 (local address 127.10.173.188:43487)
0811 22:15:07.540807 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.540811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.540824 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:15:07.540864 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.540867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.540955 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.541133 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.542206 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.543382 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.545339 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.546340 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.546542 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.551540 (+ 4998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.551547 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.551552 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.551602 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.556386 (+ 4784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.557330 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.557336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.557341 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.557528 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.558069 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.558074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.558077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.558567 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.560585 (+ 2018us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.560868 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.561225 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371}
I20260811 22:15:07.567548 407 sys_catalog.cc:455] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } } }
I20260811 22:15:07.568099 407 sys_catalog.cc:458] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:07.569032 32599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb: Bootstrap starting.
I20260811 22:15:07.570286 383 sys_catalog.cc:564] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:07.573987 32599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:07.575452 32599 log.cc:824] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:07.578884 32599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb: No bootstrap required, opened a new log
I20260811 22:15:07.591243 32599 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } }
I20260811 22:15:07.591722 32599 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:07.591884 32599 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca1b1e45dfd34b7b8180328e08a9c7eb, State: Initialized, Role: FOLLOWER
I20260811 22:15:07.592864 32599 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } }
I20260811 22:15:07.594931 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.554626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50408 (local address 127.10.173.188:43487)
0811 22:15:07.562400 (+ 7774us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.562444 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.569488 (+ 7044us) server_negotiation.cc:407] Connection header received
0811 22:15:07.569825 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.569831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.569919 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.569995 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.573061 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.574716 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.577827 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.578910 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.579034 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.585217 (+ 6183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.585228 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.585233 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.585280 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.589961 (+ 4681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.591233 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.591242 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.591248 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.591451 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.592306 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.592314 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.592320 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.592810 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.593258 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.594007 (+ 749us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.594359 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7624,"thread_start_us":15777,"threads_started":1}
I20260811 22:15:07.600281 418 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } } }
I20260811 22:15:07.600692 418 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:07.602882 32599 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:07.603833 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.554836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56796 (local address 127.10.173.190:40897)
0811 22:15:07.555043 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.555051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.566948 (+ 11897us) server_negotiation.cc:407] Connection header received
0811 22:15:07.568005 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.568014 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.568093 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.568164 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.574783 (+ 6619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.575842 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.581213 (+ 5371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.581972 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.582074 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.590489 (+ 8415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.590496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.590500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.590542 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.594614 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.599201 (+ 4587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.599210 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.599217 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.599434 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.601857 (+ 2423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.601862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.601865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.602265 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.602558 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.603213 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.603477 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:15:07.606205 32727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.554324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41742 (local address 127.10.173.189:41363)
0811 22:15:07.556696 (+ 2372us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.556703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.571962 (+ 15259us) server_negotiation.cc:407] Connection header received
0811 22:15:07.572437 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.572443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.572525 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.572599 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.578096 (+ 5497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.579191 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.585747 (+ 6556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.586800 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.586912 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.595905 (+ 8993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.595917 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.595925 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.595993 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.601274 (+ 5281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.602527 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.602537 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.602543 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.602759 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.603448 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.603457 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.603462 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.604297 (+ 835us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.604600 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.605339 (+ 739us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.605674 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2213}
W20260811 22:15:07.624922 437 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:15:07.625065 437 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:15:07.611307 32730 master_runner.cc:186] Error getting master registration for 127.10.173.190:40897: OK, instance_id { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" instance_seqno: 1786486506392244 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:15:07.627354 32730 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 22:15:07.627861 451 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:15:07.627985 451 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 22:15:07.629724 32515 tablet.cc:2369] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:07.616551 32483 master_runner.cc:186] Error getting master registration for 127.10.173.189:41363: OK, instance_id { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" instance_seqno: 1786486506916252 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:15:07.632933 32483 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:15:07.634649 407 raft_consensus.cc:482] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:07.634938 407 raft_consensus.cc:504] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } }
I20260811 22:15:07.636646 407 leader_election.cc:288] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca1b1e45dfd34b7b8180328e08a9c7eb (127.10.173.190:40897), 03c2e84d56534fda8c59ed81e638092e (127.10.173.189:41363)
I20260811 22:15:07.637727 32554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e43f05a7de3d48688d8583766f119933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" is_pre_election: true
I20260811 22:15:07.638391 32554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e43f05a7de3d48688d8583766f119933 in term 0.
I20260811 22:15:07.639108 32766 leader_election.cc:302] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca1b1e45dfd34b7b8180328e08a9c7eb, e43f05a7de3d48688d8583766f119933; no voters:
I20260811 22:15:07.639561 407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:07.639727 407 raft_consensus.cc:482] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:07.639842 407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:07.642357 32718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } attempt: 5
I20260811 22:15:07.643386 407 raft_consensus.cc:504] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } }
I20260811 22:15:07.644273 407 leader_election.cc:288] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [CANDIDATE]: Term 1 election: Requested vote from peers ca1b1e45dfd34b7b8180328e08a9c7eb (127.10.173.190:40897), 03c2e84d56534fda8c59ed81e638092e (127.10.173.189:41363)
I20260811 22:15:07.646500 32554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e43f05a7de3d48688d8583766f119933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb"
I20260811 22:15:07.646725 32554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [term 0 FOLLOWER]: Advancing to term 1
Aug 11 22:15:07 dist-test-slave-68xk krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
I20260811 22:15:07.654731 32554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e43f05a7de3d48688d8583766f119933 in term 1.
I20260811 22:15:07.657657 32766 leader_election.cc:302] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca1b1e45dfd34b7b8180328e08a9c7eb, e43f05a7de3d48688d8583766f119933; no voters:
I20260811 22:15:07.658247 407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:07.665712 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.642948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:43487 (local address 127.0.0.1:50424)
0811 22:15:07.643302 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:07.643323 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:07.643437 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:07.644400 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:07.644405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:07.644956 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:07.645191 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.645204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.646796 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.646802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.648228 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:07.648246 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.649483 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.649489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:07.649750 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.650504 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:07.650531 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:07.653423 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:07.659382 (+ 5959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.659389 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.659393 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.660446 (+ 1053us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.661325 (+ 879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:07.661330 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:07.661364 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:07.661721 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:07.663181 (+ 1460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:07.663216 (+ 35us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:07.663509 (+ 293us) client_negotiation.cc:769] Sending connection context
0811 22:15:07.663766 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 22:15:07.664382 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260811 22:15:07.667647 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:07.643695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50424 (local address 127.10.173.188:43487)
0811 22:15:07.643862 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:07.643868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:07.643885 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:15:07.643945 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:07.643949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:07.644026 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:07.644197 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:07.645361 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.646594 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.648413 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:07.649150 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:07.649282 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:07.653672 (+ 4390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:07.653684 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:07.653689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:07.653729 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:07.659199 (+ 5470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.660722 (+ 1523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.660728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.660734 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.661082 (+ 348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:07.662043 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:07.662050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:07.662054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:07.662945 (+ 891us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:07.665250 (+ 2305us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:07.665914 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 22:15:07.666696 (+ 782us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:15:07.668335 407 raft_consensus.cc:686] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [term 1 LEADER]: Becoming Leader. State: Replica: e43f05a7de3d48688d8583766f119933, State: Running, Role: LEADER
I20260811 22:15:07.669039 407 consensus_queue.cc:237] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } }
I20260811 22:15:07.673990 32718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e: Bootstrap starting.
I20260811 22:15:07.681316 32718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:07.683820 32718 log.cc:824] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:07.687451 32718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e: No bootstrap required, opened a new log
I20260811 22:15:07.688812 454 sys_catalog.cc:455] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e43f05a7de3d48688d8583766f119933. Latest consensus state: current_term: 1 leader_uuid: "e43f05a7de3d48688d8583766f119933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } } }
I20260811 22:15:07.689249 454 sys_catalog.cc:458] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [sys.catalog]: This master's current role is: LEADER
W20260811 22:15:07.692493 310 tablet.cc:2369] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:07.695013 459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:07.697427 459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:07.699350 32718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } }
I20260811 22:15:07.699800 32718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:07.699934 32718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c2e84d56534fda8c59ed81e638092e, State: Initialized, Role: FOLLOWER
I20260811 22:15:07.700763 32718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } }
I20260811 22:15:07.701781 461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } } }
I20260811 22:15:07.702277 461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:07.703349 32718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:07.703325 32676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e43f05a7de3d48688d8583766f119933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03c2e84d56534fda8c59ed81e638092e"
I20260811 22:15:07.703856 32676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:07.708456 32676 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e43f05a7de3d48688d8583766f119933 in term 1.
I20260811 22:15:07.709003 32677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e43f05a7de3d48688d8583766f119933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03c2e84d56534fda8c59ed81e638092e" is_pre_election: true
I20260811 22:15:07.709446 32677 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e43f05a7de3d48688d8583766f119933 in term 1. Re-sending same reply.
I20260811 22:15:07.740497 32677 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [term 1 FOLLOWER]: Refusing update from remote peer e43f05a7de3d48688d8583766f119933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:07.741973 454 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:07.748404 32554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [term 1 FOLLOWER]: Refusing update from remote peer e43f05a7de3d48688d8583766f119933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:07.749370 454 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:07.767448 418 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e43f05a7de3d48688d8583766f119933. Latest consensus state: current_term: 1 leader_uuid: "e43f05a7de3d48688d8583766f119933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } } }
I20260811 22:15:07.767889 418 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:07.773289 481 mvcc.cc:205] Tried to move back new op lower bound from 7317448735691337728 to 7317448735418335232. Current Snapshot: MvccSnapshot[applied={T|T < 7317448735691337728}]
I20260811 22:15:07.774534 467 mvcc.cc:205] Tried to move back new op lower bound from 7317448735691337728 to 7317448735418335232. Current Snapshot: MvccSnapshot[applied={T|T < 7317448735691337728}]
I20260811 22:15:07.805277 459 catalog_manager.cc:1347] Generated new cluster ID: 644e2bb59100458083cdd4344166072a
I20260811 22:15:07.805428 459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:07.807027 418 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e43f05a7de3d48688d8583766f119933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } } }
I20260811 22:15:07.807325 418 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:07.809173 407 sys_catalog.cc:455] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e43f05a7de3d48688d8583766f119933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } } }
I20260811 22:15:07.809477 407 sys_catalog.cc:458] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:07.815312 461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e43f05a7de3d48688d8583766f119933. Latest consensus state: current_term: 1 leader_uuid: "e43f05a7de3d48688d8583766f119933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } } }
I20260811 22:15:07.815650 461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:15:07.826939 454 sys_catalog.cc:455] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e43f05a7de3d48688d8583766f119933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } } }
I20260811 22:15:07.828379 483 mvcc.cc:205] Tried to move back new op lower bound from 7317448735691337728 to 7317448735418335232. Current Snapshot: MvccSnapshot[applied={T|T < 7317448735691337728}]
I20260811 22:15:07.828421 454 sys_catalog.cc:458] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933 [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:07.834800 461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e43f05a7de3d48688d8583766f119933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca1b1e45dfd34b7b8180328e08a9c7eb" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 40897 } } peers { permanent_uuid: "03c2e84d56534fda8c59ed81e638092e" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 41363 } } peers { permanent_uuid: "e43f05a7de3d48688d8583766f119933" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 43487 } } }
I20260811 22:15:07.835064 461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e [sys.catalog]: This master's current role is: FOLLOWER
W20260811 22:15:07.845721 494 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:15:07.845883 494 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:15:07.858625 459 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:07.862270 459 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:07.886554 459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e43f05a7de3d48688d8583766f119933: Generated new TSK 0
I20260811 22:15:07.891076 459 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:15:08.629979 451 catalog_manager.cc:1259] Loaded cluster ID: 644e2bb59100458083cdd4344166072a
I20260811 22:15:08.630132 451 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb: loading cluster ID for follower catalog manager: success
I20260811 22:15:08.632719 451 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb: acquiring CA information for follower catalog manager: success
I20260811 22:15:08.634559 451 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ca1b1e45dfd34b7b8180328e08a9c7eb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:15:08.847991 494 catalog_manager.cc:1259] Loaded cluster ID: 644e2bb59100458083cdd4344166072a
I20260811 22:15:08.848130 494 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e: loading cluster ID for follower catalog manager: success
I20260811 22:15:08.850836 494 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e: acquiring CA information for follower catalog manager: success
I20260811 22:15:08.852692 494 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 03c2e84d56534fda8c59ed81e638092e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:15:09.080688 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 32483
I20260811 22:15:09.082055 32590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:09.192660 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 32483
I20260811 22:15:09.227711 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 32608
I20260811 22:15:09.228991 32711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:09.336360 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 32608
I20260811 22:15:09.365271 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 32730
I20260811 22:15:09.366626 376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:09.477006 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 32730
2026-08-11T22:15:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5199 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:09 dist-test-slave-68xk krb5kdc[519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:09 dist-test-slave-68xk krb5kdc[519](info): set up 2 sockets
Aug 11 22:15:09 dist-test-slave-68xk krb5kdc[519](info): commencing operation
krb5kdc: starting...
W20260811 22:15:10.840853 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:15:11 dist-test-slave-68xk krb5kdc[519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:15:11.136377 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38709
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:37459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:38709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:11.478399 543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:11.478708 543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:11.478883 543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:11.486094 543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:11.486214 543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:11.486313 543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:11.486450 543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:11.486518 543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:11.496407 543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:38709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:11.497817 543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:11.499221 543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:11.511700 555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:11.511783 552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:11.511797 553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:11.516251 543 server_base.cc:1034] running on GCE node
I20260811 22:15:11.517261 543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:11.519044 543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:11.520337 543 hybrid_clock.cc:648] HybridClock initialized: now 1786486511520228 us; error 89 us; skew 500 ppm
I20260811 22:15:11.520751 543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:11 dist-test-slave-68xk krb5kdc[519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486511, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:11.529968 543 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:15:11.531869 543 webserver.cc:460] Webserver started at http://127.10.173.190:33359/ using document root <none> and password file <none>
I20260811 22:15:11.532698 543 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:11.532897 543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:11.533210 543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:11.536558 543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "43bd63bdc79942f999f8e21d653cf66b"
format_stamp: "Formatted at 2026-08-11 22:15:11 on dist-test-slave-68xk"
server_key: "23b4e82e9c3df2e4a64d811d1d5e2a8b"
server_key_iv: "336722854d9b682d9ea875ae47877806"
server_key_version: "encryptionkey@0"
I20260811 22:15:11.537360 543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "43bd63bdc79942f999f8e21d653cf66b"
format_stamp: "Formatted at 2026-08-11 22:15:11 on dist-test-slave-68xk"
server_key: "23b4e82e9c3df2e4a64d811d1d5e2a8b"
server_key_iv: "336722854d9b682d9ea875ae47877806"
server_key_version: "encryptionkey@0"
I20260811 22:15:11.543166 543 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 22:15:11.546926 573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:11.548300 543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:15:11.548508 543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "43bd63bdc79942f999f8e21d653cf66b"
format_stamp: "Formatted at 2026-08-11 22:15:11 on dist-test-slave-68xk"
server_key: "23b4e82e9c3df2e4a64d811d1d5e2a8b"
server_key_iv: "336722854d9b682d9ea875ae47877806"
server_key_version: "encryptionkey@0"
I20260811 22:15:11.548720 543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:11.575054 543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:11.580248 543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:11.580602 543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:11.602927 543 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:38709
I20260811 22:15:11.602953 663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:38709 every 8 connection(s)
I20260811 22:15:11.604645 543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:11.608755 665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:11.614787 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 543
I20260811 22:15:11.615093 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:15:11.615877 10934 external_mini_cluster.cc:1428] Setting key 099ec204b617d8ce8c67ab37377400a1
I20260811 22:15:11.619587 665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b: Bootstrap starting.
I20260811 22:15:11.623306 665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:11.624552 665 log.cc:824] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:11.628068 665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b: No bootstrap required, opened a new log
Aug 11 22:15:11 dist-test-slave-68xk krb5kdc[519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486511, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:11.638947 665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bd63bdc79942f999f8e21d653cf66b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38709 } }
I20260811 22:15:11.639472 665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:11.639577 665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43bd63bdc79942f999f8e21d653cf66b, State: Initialized, Role: FOLLOWER
I20260811 22:15:11.640266 665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bd63bdc79942f999f8e21d653cf66b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38709 } }
I20260811 22:15:11.640534 665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:11.640707 665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:11.640909 665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:11.643090 677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:11.617888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.10.173.190:38709)
0811 22:15:11.618808 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:11.618821 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:11.618867 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:15:11.620856 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:11.620884 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:11.621312 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:11.621668 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:11.623818 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.625747 (+ 1929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.627743 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:11.628743 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:11.628902 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:11.633985 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:11.633996 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:11.634016 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:11.634068 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:11.638256 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:11.639097 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:11.639105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:11.639113 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:11.639331 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:11.639855 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:11.639859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:11.639861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:11.640394 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:11.640765 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:11.641193 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 22:15:11.641663 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":482,"threads_started":1}
I20260811 22:15:11.644368 665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bd63bdc79942f999f8e21d653cf66b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38709 } }
I20260811 22:15:11.644903 665 leader_election.cc:302] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43bd63bdc79942f999f8e21d653cf66b; no voters:
I20260811 22:15:11.645632 665 leader_election.cc:288] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:11.645771 680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:11.647028 680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [term 1 LEADER]: Becoming Leader. State: Replica: 43bd63bdc79942f999f8e21d653cf66b, State: Running, Role: LEADER
I20260811 22:15:11.647544 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:15:11.647692 680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bd63bdc79942f999f8e21d653cf66b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38709 } }
I20260811 22:15:11.648005 665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:11.650455 682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43bd63bdc79942f999f8e21d653cf66b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bd63bdc79942f999f8e21d653cf66b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38709 } } }
I20260811 22:15:11.650429 684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43bd63bdc79942f999f8e21d653cf66b. Latest consensus state: current_term: 1 leader_uuid: "43bd63bdc79942f999f8e21d653cf66b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bd63bdc79942f999f8e21d653cf66b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38709 } } }
I20260811 22:15:11.650867 684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:11.650867 682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:11.654332 688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:11.659552 688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:11.668502 688 catalog_manager.cc:1347] Generated new cluster ID: ae971365da0744378072d2acf741ca89
I20260811 22:15:11.668627 688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:11.702373 688 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:11.704157 688 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:11.714260 688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43bd63bdc79942f999f8e21d653cf66b: Generated new TSK 0
I20260811 22:15:11.715126 688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:15:11.762205 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 543
I20260811 22:15:11.763592 658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:11.864799 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 543
2026-08-11T22:15:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2387 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:11 dist-test-slave-68xk krb5kdc[726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:11 dist-test-slave-68xk krb5kdc[726](info): set up 2 sockets
Aug 11 22:15:11 dist-test-slave-68xk krb5kdc[726](info): commencing operation
krb5kdc: starting...
W20260811 22:15:13.311414 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.398s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:15:13 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:15:13.626474 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38319
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:38319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:13.984274 752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:13.984575 752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:13.984737 752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:13.992235 752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:13.992348 752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:13.992435 752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:13.992553 752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:13.992616 752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:14.002805 752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:38319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:14.004266 752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:14.005759 752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:14.018704 764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:14.018803 761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:14.018827 759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:14.020026 752 server_base.cc:1034] running on GCE node
I20260811 22:15:14.020813 752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:14.022291 752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:14.023579 752 hybrid_clock.cc:648] HybridClock initialized: now 1786486514023513 us; error 45 us; skew 500 ppm
I20260811 22:15:14.023984 752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:14 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486514, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:14.027501 752 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:15:14.029074 752 webserver.cc:460] Webserver started at http://127.10.173.190:34677/ using document root <none> and password file <none>
I20260811 22:15:14.029814 752 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:14.029971 752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:14.030303 752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:14.033401 752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "daa57e0e4bf346e29bf305e63136797b"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-68xk"
server_key: "4b1765d492f85902e48b493c7dbb32e5"
server_key_iv: "1c6cecbcd037bb31a9a0e1015ce87e8b"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.034205 752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "daa57e0e4bf346e29bf305e63136797b"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-68xk"
server_key: "4b1765d492f85902e48b493c7dbb32e5"
server_key_iv: "1c6cecbcd037bb31a9a0e1015ce87e8b"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.039641 752 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 22:15:14.043129 772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:14.044464 752 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:14.044653 752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "daa57e0e4bf346e29bf305e63136797b"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-68xk"
server_key: "4b1765d492f85902e48b493c7dbb32e5"
server_key_iv: "1c6cecbcd037bb31a9a0e1015ce87e8b"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.044859 752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:14.058784 752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:14.062994 752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:14.063318 752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:14.083042 752 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:38319
I20260811 22:15:14.083092 843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:38319 every 8 connection(s)
I20260811 22:15:14.084506 752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:14.088304 844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:14.094034 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 752
I20260811 22:15:14.094280 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:15:14.094977 10934 external_mini_cluster.cc:1428] Setting key 613d4ffeb8d27328cea16316579118cf
I20260811 22:15:14.099112 844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Bootstrap starting.
I20260811 22:15:14.102941 844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:14.104058 844 log.cc:824] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:14.107137 844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: No bootstrap required, opened a new log
Aug 11 22:15:14 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:14.116032 844 raft_consensus.cc:348] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:14.116429 844 raft_consensus.cc:374] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:14.116559 844 raft_consensus.cc:729] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daa57e0e4bf346e29bf305e63136797b, State: Initialized, Role: FOLLOWER
I20260811 22:15:14.117197 844 consensus_queue.cc:260] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:14.117442 844 raft_consensus.cc:388] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:14.117609 844 raft_consensus.cc:482] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:14.117812 844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:14.119130 849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.097092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.10.173.190:38319)
0811 22:15:14.097918 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.097930 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.097972 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:15:14.099796 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.099820 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.100185 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.100500 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:14.102385 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.103829 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.105526 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.106171 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.106310 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.111358 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:14.111366 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:14.111379 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:14.111414 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:14.114763 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.115538 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.115544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.115550 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.115715 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.116125 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.116130 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.116135 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.116811 (+ 676us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:14.117147 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.117612 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.118050 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":407,"threads_started":1}
I20260811 22:15:14.120829 844 raft_consensus.cc:504] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:14.121341 844 leader_election.cc:302] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daa57e0e4bf346e29bf305e63136797b; no voters:
I20260811 22:15:14.122008 844 leader_election.cc:288] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:14.122161 851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:14.122979 851 raft_consensus.cc:686] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 1 LEADER]: Becoming Leader. State: Replica: daa57e0e4bf346e29bf305e63136797b, State: Running, Role: LEADER
I20260811 22:15:14.123560 851 consensus_queue.cc:237] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:14.124118 844 sys_catalog.cc:564] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:14.126394 852 sys_catalog.cc:455] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daa57e0e4bf346e29bf305e63136797b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } } }
I20260811 22:15:14.126816 852 sys_catalog.cc:458] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:14.126899 854 sys_catalog.cc:455] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: SysCatalogTable state changed. Reason: New leader daa57e0e4bf346e29bf305e63136797b. Latest consensus state: current_term: 1 leader_uuid: "daa57e0e4bf346e29bf305e63136797b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } } }
I20260811 22:15:14.127220 854 sys_catalog.cc:458] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:14.127843 869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:14.133957 869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:14.143340 869 catalog_manager.cc:1347] Generated new cluster ID: 22f3e66a9d394596930f325132324e48
I20260811 22:15:14.143571 869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:14.184389 869 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:14.186367 869 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:14.193591 869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Generated new TSK 0
I20260811 22:15:14.194674 869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:15:14.266682 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38319
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:14.610178 892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:14.610569 892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:14.610911 892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:14.619449 892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:14.619608 892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:14.619814 892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:15:14.630592 892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:14.631906 892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:14.633220 892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:14.647420 901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:14.647472 904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:14.647517 907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:14.648643 892 server_base.cc:1034] running on GCE node
I20260811 22:15:14.649295 892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:14.650380 892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:14.651710 892 hybrid_clock.cc:648] HybridClock initialized: now 1786486514651651 us; error 62 us; skew 500 ppm
I20260811 22:15:14.652077 892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:14 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486514, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:14.655486 892 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:15:14.657058 892 webserver.cc:460] Webserver started at http://127.10.173.129:35633/ using document root <none> and password file <none>
I20260811 22:15:14.657829 892 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:14.657987 892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:14.658337 892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:14.661319 892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "d5168b81aebe4fb89a48ebcf5a930b78"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-68xk"
server_key: "da207de210b7a7344a28c5502a07c665"
server_key_iv: "2724aac1669697d846ad34d3f54c415a"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.662128 892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "d5168b81aebe4fb89a48ebcf5a930b78"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-68xk"
server_key: "da207de210b7a7344a28c5502a07c665"
server_key_iv: "2724aac1669697d846ad34d3f54c415a"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.667490 892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:15:14.671080 915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:14.672904 892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:14.673096 892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "d5168b81aebe4fb89a48ebcf5a930b78"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-68xk"
server_key: "da207de210b7a7344a28c5502a07c665"
server_key_iv: "2724aac1669697d846ad34d3f54c415a"
server_key_version: "encryptionkey@0"
I20260811 22:15:14.673319 892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:14.692694 892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:14.697459 892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:14.697768 892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:14.698829 892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:14.700248 892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:14.700354 892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:14.700505 892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:14.700580 892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:14.735208 892 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:45097
I20260811 22:15:14.735258 1092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:45097 every 8 connection(s)
I20260811 22:15:14.736536 892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:14.746940 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 892
I20260811 22:15:14.747211 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:15:14.748054 10934 external_mini_cluster.cc:1428] Setting key f00a57c83a9d8d1e6002ef7a002dec4f
Aug 11 22:15:14 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486514, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:14.757103 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.739206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:43661 (local address 127.10.173.190:38319)
0811 22:15:14.739654 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:14.739658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:14.740620 (+ 962us) server_negotiation.cc:407] Connection header received
0811 22:15:14.742480 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:14.742485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:14.742578 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:14.742705 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:14.744282 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.745022 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.746724 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.747178 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.747331 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.751216 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:14.751223 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:14.751226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:14.751259 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:14.753865 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.754712 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.754727 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.754731 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.754862 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:14.755327 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:14.755332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:14.755337 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:14.755606 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:14.755778 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:14.756485 (+ 707us) server_negotiation.cc:299] Negotiation successful
0811 22:15:14.756714 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":231,"threads_started":1}
I20260811 22:15:14.758041 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:14.739739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:43661)
0811 22:15:14.740441 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:14.740486 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:14.742182 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:14.742962 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:14.742971 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:14.743510 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:14.744087 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.744104 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.745218 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.745229 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.746573 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:14.746585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:14.747430 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:14.747441 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:14.747684 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:14.748457 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:14.748485 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:14.750989 (+ 2504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:14.754017 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.754026 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.754042 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.754591 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.754982 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:14.754987 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:14.754991 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:14.755191 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:14.755743 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:14.755751 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:14.756095 (+ 344us) client_negotiation.cc:769] Sending connection context
0811 22:15:14.756374 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 22:15:14.756908 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":268,"threads_started":1}
I20260811 22:15:14.759840 1093 heartbeater.cc:347] Connected to a master server at 127.10.173.190:38319
I20260811 22:15:14.760177 1093 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:14.760916 1093 heartbeater.cc:511] Master 127.10.173.190:38319 requested a full tablet report, sending...
I20260811 22:15:14.763319 795 ts_manager.cc:194] Registered new tserver with Master: d5168b81aebe4fb89a48ebcf5a930b78 (127.10.173.129:45097)
I20260811 22:15:14.765594 795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:43661
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:15:14.818813 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38319
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:15.168692 1102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:15.169030 1102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:15.169198 1102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:15.176783 1102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:15.176949 1102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:15.177122 1102 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:15:15.187316 1102 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:15.188673 1102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:15.190033 1102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:15.204681 1112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:15.204712 1114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:15.204758 1116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:15.205672 1102 server_base.cc:1034] running on GCE node
I20260811 22:15:15.206341 1102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:15.207409 1102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:15.208633 1102 hybrid_clock.cc:648] HybridClock initialized: now 1786486515208599 us; error 29 us; skew 500 ppm
I20260811 22:15:15.209012 1102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:15 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486515, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:15.212446 1102 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:15:15.214138 1102 webserver.cc:460] Webserver started at http://127.10.173.130:36743/ using document root <none> and password file <none>
I20260811 22:15:15.214972 1102 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:15.215121 1102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:15.215423 1102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:15.218271 1102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "3469cb92c95a455c892fe73f075e6ffc"
format_stamp: "Formatted at 2026-08-11 22:15:15 on dist-test-slave-68xk"
server_key: "f6ec04255bc5310a7060569cfe6e499d"
server_key_iv: "c831cb69186889439d9cfdb9fa032628"
server_key_version: "encryptionkey@0"
I20260811 22:15:15.219098 1102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "3469cb92c95a455c892fe73f075e6ffc"
format_stamp: "Formatted at 2026-08-11 22:15:15 on dist-test-slave-68xk"
server_key: "f6ec04255bc5310a7060569cfe6e499d"
server_key_iv: "c831cb69186889439d9cfdb9fa032628"
server_key_version: "encryptionkey@0"
I20260811 22:15:15.225500 1102 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 22:15:15.229293 1124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:15.230736 1102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:15.230909 1102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "3469cb92c95a455c892fe73f075e6ffc"
format_stamp: "Formatted at 2026-08-11 22:15:15 on dist-test-slave-68xk"
server_key: "f6ec04255bc5310a7060569cfe6e499d"
server_key_iv: "c831cb69186889439d9cfdb9fa032628"
server_key_version: "encryptionkey@0"
I20260811 22:15:15.231082 1102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:15.242735 1102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:15.246641 1102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:15.246927 1102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:15.247875 1102 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:15.249315 1102 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:15.249389 1102 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:15.249531 1102 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:15.249598 1102 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:15.286290 1102 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:32805
I20260811 22:15:15.286343 1308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:32805 every 8 connection(s)
I20260811 22:15:15.287649 1102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:15.297788 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 1102
I20260811 22:15:15.298022 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:15:15.298614 10934 external_mini_cluster.cc:1428] Setting key dcc62e0f71ef1b205a4a7cb6d44463b7
Aug 11 22:15:15 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486515, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:15.308907 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.290251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:60227 (local address 127.10.173.190:38319)
0811 22:15:15.290753 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.290758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.291632 (+ 874us) server_negotiation.cc:407] Connection header received
0811 22:15:15.293487 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.293492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.293583 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.293706 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:15.295354 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.296098 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.297663 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.298179 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.298313 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.302131 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:15.302138 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:15.302141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:15.302175 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:15.305300 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:15.306056 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:15.306061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:15.306065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:15.306256 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:15.306906 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:15.306910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:15.306913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:15.307237 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:15.307373 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.308384 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.308612 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":276,"threads_started":1}
I20260811 22:15:15.310442 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.290711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:60227)
0811 22:15:15.291437 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:15.291477 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:15.293190 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:15.293965 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:15.293973 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:15.294579 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:15.295154 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.295172 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.296273 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.296277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.297506 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.297524 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.298407 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.298445 (+ 38us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.298674 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.299465 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:15.299497 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:15.301897 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:15.305453 (+ 3556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:15.305463 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:15.305480 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:15.305920 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:15.306435 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:15.306443 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:15.306449 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:15.306726 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:15.307388 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:15.307400 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:15.307906 (+ 506us) client_negotiation.cc:769] Sending connection context
0811 22:15:15.308235 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 22:15:15.308858 (+ 623us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":286,"threads_started":1}
I20260811 22:15:15.312229 1309 heartbeater.cc:347] Connected to a master server at 127.10.173.190:38319
I20260811 22:15:15.312608 1309 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:15.313659 1309 heartbeater.cc:511] Master 127.10.173.190:38319 requested a full tablet report, sending...
I20260811 22:15:15.315440 795 ts_manager.cc:194] Registered new tserver with Master: 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805)
I20260811 22:15:15.316313 795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:60227
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:15:15.366252 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38319
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:15.710330 1317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:15.710757 1317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:15.711050 1317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:15.721406 1317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:15.721516 1317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:15.721714 1317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:15:15.732137 1317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:15.733462 1317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:15.735038 1317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:15.749975 1327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:15.750017 1325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:15.750205 1322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:15.751818 1317 server_base.cc:1034] running on GCE node
I20260811 22:15:15.752483 1317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:15.753461 1317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:15.754685 1317 hybrid_clock.cc:648] HybridClock initialized: now 1786486515754638 us; error 36 us; skew 500 ppm
I20260811 22:15:15.755050 1317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:15 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486515, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:15.758653 1317 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:15:15.760196 1317 webserver.cc:460] Webserver started at http://127.10.173.131:33199/ using document root <none> and password file <none>
I20260811 22:15:15.760946 1317 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:15.761126 1317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:15.761458 1317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:15.764777 1317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "df6e68db6f6a4a699e2282f05fb32b5d"
format_stamp: "Formatted at 2026-08-11 22:15:15 on dist-test-slave-68xk"
server_key: "db5ef7df951946dd0d35078b1de48c86"
server_key_iv: "42917e676a88144b75745b850e8896ea"
server_key_version: "encryptionkey@0"
I20260811 22:15:15.765684 1317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "df6e68db6f6a4a699e2282f05fb32b5d"
format_stamp: "Formatted at 2026-08-11 22:15:15 on dist-test-slave-68xk"
server_key: "db5ef7df951946dd0d35078b1de48c86"
server_key_iv: "42917e676a88144b75745b850e8896ea"
server_key_version: "encryptionkey@0"
I20260811 22:15:15.768848 1093 heartbeater.cc:503] Master 127.10.173.190:38319 was elected leader, sending a full tablet report...
I20260811 22:15:15.771835 1317 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 22:15:15.775478 1339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:15.776991 1317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:15:15.777186 1317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "df6e68db6f6a4a699e2282f05fb32b5d"
format_stamp: "Formatted at 2026-08-11 22:15:15 on dist-test-slave-68xk"
server_key: "db5ef7df951946dd0d35078b1de48c86"
server_key_iv: "42917e676a88144b75745b850e8896ea"
server_key_version: "encryptionkey@0"
I20260811 22:15:15.777400 1317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:15.809019 1317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:15.813704 1317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:15.814018 1317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:15.815130 1317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:15.816768 1317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:15.816874 1317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:15.817054 1317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:15.817130 1317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:15.855099 1317 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:40483
I20260811 22:15:15.855134 1525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:40483 every 8 connection(s)
I20260811 22:15:15.856568 1317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:15.863929 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 1317
I20260811 22:15:15.864157 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:15:15.864797 10934 external_mini_cluster.cc:1428] Setting key f174ddf5bf336cf7271f2da137cea6ac
Aug 11 22:15:15 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486515, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:15.878173 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.859409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:55557 (local address 127.10.173.190:38319)
0811 22:15:15.859913 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.859917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.860752 (+ 835us) server_negotiation.cc:407] Connection header received
0811 22:15:15.862689 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.862695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.862793 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.862919 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:15.865019 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.866007 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.868342 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.868818 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.868921 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.872492 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:15.872497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:15.872501 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:15.872548 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:15.875374 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:15.876093 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:15.876097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:15.876099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:15.876244 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:15.876631 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:15.876634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:15.876636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:15.876897 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:15.877014 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.877729 (+ 715us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.877904 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":282,"threads_started":1}
I20260811 22:15:15.879451 1531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.859847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:55557)
0811 22:15:15.860565 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:15.860611 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:15.862343 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:15.863178 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:15.863186 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:15.863927 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:15.864808 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.864841 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.866191 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.866195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.868141 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.868158 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.869024 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.869028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.869233 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.869907 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:15.869935 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:15.872241 (+ 2306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:15.875513 (+ 3272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:15.875520 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:15.875533 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:15.875974 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:15.876344 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:15.876347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:15.876349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:15.876527 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:15.877008 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:15.877014 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:15.877380 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 22:15:15.877614 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 22:15:15.878078 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":321,"threads_started":1}
I20260811 22:15:15.881265 1528 heartbeater.cc:347] Connected to a master server at 127.10.173.190:38319
I20260811 22:15:15.881609 1528 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:15.882455 1528 heartbeater.cc:511] Master 127.10.173.190:38319 requested a full tablet report, sending...
I20260811 22:15:15.883994 794 ts_manager.cc:194] Registered new tserver with Master: df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483)
I20260811 22:15:15.884951 794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:55557
I20260811 22:15:15.893796 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:15:15.911419 1530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.898629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.10.173.190:38319)
0811 22:15:15.898823 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.898827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.899289 (+ 462us) server_negotiation.cc:407] Connection header received
0811 22:15:15.899430 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.899434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.899500 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.899587 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:15.901023 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.901808 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.903337 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.903821 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.903943 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.906338 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:15.906343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:15.906346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:15.906382 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:15.908983 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:15.909683 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:15.909687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:15.909689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:15.909821 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:15.910151 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:15.910154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:15.910156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:15.910516 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:15.910699 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.911016 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.911187 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:15:15.916093 794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:15:15.923103 794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:15.949421 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.937289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.10.173.130:32805)
0811 22:15:15.937965 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.937977 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.938632 (+ 655us) server_negotiation.cc:407] Connection header received
0811 22:15:15.939019 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.939031 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.939313 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.939490 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:15.940675 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.942839 (+ 2164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.946505 (+ 3666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.948045 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.948221 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.948392 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.948823 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.949023 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":334,"threads_started":1}
I20260811 22:15:15.949990 1543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.937628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:40483 (local address 127.0.0.1:58216)
0811 22:15:15.938526 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:15.938547 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:15.938734 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:15.940371 (+ 1637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:15.940378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:15.940411 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:15.940954 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.940969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.944033 (+ 3064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.944040 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.946910 (+ 2870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.946926 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.948826 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.948832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.949224 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.949264 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:15:15.949350 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:15:15.949537 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"thread_start_us":956,"threads_started":1}
I20260811 22:15:15.950511 1545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.937136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:32805 (local address 127.0.0.1:41752)
0811 22:15:15.938454 (+ 1318us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:15.938497 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:15.938765 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:15.939843 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:15.939850 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:15.939890 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:15.940385 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.940403 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.943081 (+ 2678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.943088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.946273 (+ 3185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.946287 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.948269 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.948275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.948504 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.948572 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 22:15:15.949810 (+ 1238us) client_negotiation.cc:240] Negotiation successful
0811 22:15:15.949984 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"mutex_wait_us":661,"thread_start_us":1059,"threads_started":1}
I20260811 22:15:15.951134 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.938281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58216 (local address 127.10.173.131:40483)
0811 22:15:15.939597 (+ 1316us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.939610 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.939651 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:15:15.939776 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.939787 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.940097 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.941190 (+ 1093us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:15.941532 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.943826 (+ 2294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.947112 (+ 3286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.948611 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.948823 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.948985 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.950587 (+ 1602us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.950786 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1079,"thread_start_us":894,"threads_started":1}
I20260811 22:15:15.954618 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.939081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45097 (local address 127.0.0.1:55286)
0811 22:15:15.946773 (+ 7692us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:15.946793 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:15.946933 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:15.947526 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:15.947531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:15.947562 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:15.947978 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.947992 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.950616 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.950620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.952499 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:15.952511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.953801 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.953815 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:15.953982 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.954017 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:15:15.954129 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:15:15.954272 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7542,"thread_start_us":7372,"threads_started":1}
I20260811 22:15:15.954770 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:15.939740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55286 (local address 127.10.173.129:45097)
0811 22:15:15.945152 (+ 5412us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:15.945162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:15.946936 (+ 1774us) server_negotiation.cc:407] Connection header received
0811 22:15:15.947098 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:15.947105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:15.947317 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:15.947542 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:15.948176 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.950466 (+ 2290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.952650 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:15.953581 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:15.953801 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:15.953965 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:15.954172 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 22:15:15.954387 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5201,"thread_start_us":5040,"threads_started":1}
I20260811 22:15:15.955199 1417 tablet_service.cc:1467] Processing CreateTablet for tablet 89f95258561a41c19711632fd3a85e90 (DEFAULT_TABLE table=test-table [id=804dc97399ea48fc86ce7c6715bf1c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:15.955199 1203 tablet_service.cc:1467] Processing CreateTablet for tablet 89f95258561a41c19711632fd3a85e90 (DEFAULT_TABLE table=test-table [id=804dc97399ea48fc86ce7c6715bf1c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:15.956892 1417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89f95258561a41c19711632fd3a85e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:15.956894 1203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89f95258561a41c19711632fd3a85e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:15.960495 987 tablet_service.cc:1467] Processing CreateTablet for tablet 89f95258561a41c19711632fd3a85e90 (DEFAULT_TABLE table=test-table [id=804dc97399ea48fc86ce7c6715bf1c8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:15.961673 987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89f95258561a41c19711632fd3a85e90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:15.972481 1555 tablet_bootstrap.cc:492] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: Bootstrap starting.
I20260811 22:15:15.973526 1556 tablet_bootstrap.cc:492] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: Bootstrap starting.
I20260811 22:15:15.974321 1557 tablet_bootstrap.cc:492] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: Bootstrap starting.
I20260811 22:15:15.977391 1555 tablet_bootstrap.cc:654] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:15.977986 1557 tablet_bootstrap.cc:654] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:15.978754 1555 log.cc:824] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:15.979269 1557 log.cc:824] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:15.980098 1556 tablet_bootstrap.cc:654] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:15.981338 1556 log.cc:824] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:15.984431 1556 tablet_bootstrap.cc:492] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: No bootstrap required, opened a new log
I20260811 22:15:15.984710 1556 ts_tablet_manager.cc:1397] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 22:15:15.985059 1557 tablet_bootstrap.cc:492] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: No bootstrap required, opened a new log
I20260811 22:15:15.985066 1555 tablet_bootstrap.cc:492] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: No bootstrap required, opened a new log
I20260811 22:15:15.985352 1555 ts_tablet_manager.cc:1397] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20260811 22:15:15.985352 1557 ts_tablet_manager.cc:1397] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260811 22:15:15.991838 1556 raft_consensus.cc:348] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } } peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } }
I20260811 22:15:15.992200 1556 raft_consensus.cc:374] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:15.992314 1556 raft_consensus.cc:729] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df6e68db6f6a4a699e2282f05fb32b5d, State: Initialized, Role: FOLLOWER
I20260811 22:15:15.992933 1556 consensus_queue.cc:260] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } } peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } }
I20260811 22:15:15.994050 1528 heartbeater.cc:503] Master 127.10.173.190:38319 was elected leader, sending a full tablet report...
I20260811 22:15:15.994724 1556 ts_tablet_manager.cc:1428] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: Time spent starting tablet: real 0.010s user 0.006s sys 0.006s
I20260811 22:15:15.997282 1555 raft_consensus.cc:348] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } } peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } }
I20260811 22:15:15.997933 1555 raft_consensus.cc:374] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:15.998063 1555 raft_consensus.cc:729] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5168b81aebe4fb89a48ebcf5a930b78, State: Initialized, Role: FOLLOWER
I20260811 22:15:15.998512 1557 raft_consensus.cc:348] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } } peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } }
I20260811 22:15:15.999008 1557 raft_consensus.cc:374] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:15.999194 1557 raft_consensus.cc:729] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3469cb92c95a455c892fe73f075e6ffc, State: Initialized, Role: FOLLOWER
I20260811 22:15:15.998837 1555 consensus_queue.cc:260] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } } peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } }
I20260811 22:15:16.000046 1557 consensus_queue.cc:260] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } } peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } }
I20260811 22:15:16.001248 1309 heartbeater.cc:503] Master 127.10.173.190:38319 was elected leader, sending a full tablet report...
I20260811 22:15:16.001574 1555 ts_tablet_manager.cc:1428] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260811 22:15:16.001981 1557 ts_tablet_manager.cc:1428] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
W20260811 22:15:16.039835 1310 tablet.cc:2369] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:15:16.108443 1529 tablet.cc:2369] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:16.192802 1570 raft_consensus.cc:482] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:16.193174 1570 raft_consensus.cc:504] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } } peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } }
I20260811 22:15:16.195142 1570 leader_election.cc:288] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805), df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483)
I20260811 22:15:16.204809 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:16.195986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:42039 (local address 127.10.173.131:40483)
0811 22:15:16.196257 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:16.196266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:16.196957 (+ 691us) server_negotiation.cc:407] Connection header received
0811 22:15:16.197127 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:16.197132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:16.197206 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:16.197309 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:16.198069 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.199228 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.202371 (+ 3143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.203634 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.203802 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:16.203878 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:16.204399 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 22:15:16.204531 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 22:15:16.204802 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:16.195539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:44917 (local address 127.10.173.130:32805)
0811 22:15:16.195829 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:16.195839 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:16.196672 (+ 833us) server_negotiation.cc:407] Connection header received
0811 22:15:16.196929 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:16.196938 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:16.197028 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:16.197144 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:16.197966 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.199158 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.202379 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.203634 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.203798 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:16.203878 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:16.204400 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 22:15:16.204534 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:15:16.205118 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:16.195652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:40483 (local address 127.10.173.129:42039)
0811 22:15:16.196850 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:16.196865 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:16.196981 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:16.197360 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:16.197367 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:16.197395 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:16.197905 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:16.197918 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.199359 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.199367 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:16.202169 (+ 2802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:16.202186 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.203951 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.203958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:16.204207 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:16.204272 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 22:15:16.204358 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:15:16.204556 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":344,"thread_start_us":845,"threads_started":1}
I20260811 22:15:16.205152 1573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:16.195379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:32805 (local address 127.10.173.129:44917)
0811 22:15:16.196548 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:16.196567 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:16.196762 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:16.197160 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:16.197163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:16.197188 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:16.197784 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:16.197804 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.199330 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.199337 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:16.202169 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:16.202186 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.203926 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.203938 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:16.204210 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:16.204252 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 22:15:16.204358 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:15:16.204553 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"mutex_wait_us":380,"thread_start_us":741,"threads_started":1}
I20260811 22:15:16.206143 1456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f95258561a41c19711632fd3a85e90" candidate_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" is_pre_election: true
I20260811 22:15:16.206156 1238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f95258561a41c19711632fd3a85e90" candidate_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3469cb92c95a455c892fe73f075e6ffc" is_pre_election: true
I20260811 22:15:16.206694 1456 raft_consensus.cc:2454] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5168b81aebe4fb89a48ebcf5a930b78 in term 0.
I20260811 22:15:16.206679 1238 raft_consensus.cc:2454] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5168b81aebe4fb89a48ebcf5a930b78 in term 0.
I20260811 22:15:16.207392 923 leader_election.cc:302] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3469cb92c95a455c892fe73f075e6ffc, d5168b81aebe4fb89a48ebcf5a930b78, df6e68db6f6a4a699e2282f05fb32b5d; no voters:
I20260811 22:15:16.207834 1570 raft_consensus.cc:2781] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:16.207981 1570 raft_consensus.cc:482] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:16.208102 1570 raft_consensus.cc:3037] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:16.210480 1570 raft_consensus.cc:504] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } } peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } }
I20260811 22:15:16.211156 1570 leader_election.cc:288] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 1 election: Requested vote from peers 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805), df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483)
I20260811 22:15:16.211647 1456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f95258561a41c19711632fd3a85e90" candidate_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6e68db6f6a4a699e2282f05fb32b5d"
I20260811 22:15:16.211659 1238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89f95258561a41c19711632fd3a85e90" candidate_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3469cb92c95a455c892fe73f075e6ffc"
I20260811 22:15:16.211866 1238 raft_consensus.cc:3037] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:16.211862 1456 raft_consensus.cc:3037] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:16.214115 1238 raft_consensus.cc:2454] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5168b81aebe4fb89a48ebcf5a930b78 in term 1.
I20260811 22:15:16.214121 1456 raft_consensus.cc:2454] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5168b81aebe4fb89a48ebcf5a930b78 in term 1.
I20260811 22:15:16.214563 923 leader_election.cc:302] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3469cb92c95a455c892fe73f075e6ffc, d5168b81aebe4fb89a48ebcf5a930b78; no voters:
I20260811 22:15:16.214859 1570 raft_consensus.cc:2781] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:16.215317 1570 raft_consensus.cc:686] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 LEADER]: Becoming Leader. State: Replica: d5168b81aebe4fb89a48ebcf5a930b78, State: Running, Role: LEADER
I20260811 22:15:16.215903 1570 consensus_queue.cc:237] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } } peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } }
I20260811 22:15:16.221086 795 catalog_manager.cc:5495] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5168b81aebe4fb89a48ebcf5a930b78 (127.10.173.129). New cstate: current_term: 1 leader_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } health_report { overall_health: UNKNOWN } } }
W20260811 22:15:16.239022 1095 tablet.cc:2369] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:16.247318 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:16.241548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.10.173.129:45097)
0811 22:15:16.241790 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:16.241795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:16.241813 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:15:16.241995 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:16.241998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:16.242072 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:16.242193 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:16.242625 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.243350 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.245056 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.245477 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.245576 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:16.246097 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:16.246684 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:16.246799 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:16.246930 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 22:15:16.246998 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:15:16.255970 1456 raft_consensus.cc:1261] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d [term 1 FOLLOWER]: Refusing update from remote peer d5168b81aebe4fb89a48ebcf5a930b78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:16.255968 1238 raft_consensus.cc:1261] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc [term 1 FOLLOWER]: Refusing update from remote peer d5168b81aebe4fb89a48ebcf5a930b78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:16.257098 1577 consensus_queue.cc:1035] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:16.257462 1570 consensus_queue.cc:1035] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:16.289779 794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:16.290092 794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:16.293902 794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Sending DeleteTablet for 3 replicas of tablet 89f95258561a41c19711632fd3a85e90
I20260811 22:15:16.295292 987 tablet_service.cc:1514] Processing DeleteTablet for tablet 89f95258561a41c19711632fd3a85e90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:16 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:55286
I20260811 22:15:16.295383 1417 tablet_service.cc:1514] Processing DeleteTablet for tablet 89f95258561a41c19711632fd3a85e90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:16 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:58216
I20260811 22:15:16.295892 1203 tablet_service.cc:1514] Processing DeleteTablet for tablet 89f95258561a41c19711632fd3a85e90 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:16 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:41752
I20260811 22:15:16.297137 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 892
I20260811 22:15:16.303028 1608 tablet_replica.cc:330] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: stopping tablet replica
I20260811 22:15:16.303669 1608 raft_consensus.cc:2229] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:16.304411 1608 raft_consensus.cc:2258] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:16.305263 1612 tablet_replica.cc:330] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: stopping tablet replica
I20260811 22:15:16.305652 1612 raft_consensus.cc:2229] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:16.306164 1612 raft_consensus.cc:2258] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:16.310374 1611 tablet_replica.cc:330] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: stopping tablet replica
I20260811 22:15:16.310956 1611 raft_consensus.cc:2229] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:16.311511 1611 raft_consensus.cc:2258] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:16.313630 1612 ts_tablet_manager.cc:1905] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:16.313764 1611 ts_tablet_manager.cc:1905] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:16.315573 1608 ts_tablet_manager.cc:1905] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 22:15:16 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:15:16.321579 1611 ts_tablet_manager.cc:1918] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:16.321820 1611 log.cc:1196] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/89f95258561a41c19711632fd3a85e90
I20260811 22:15:16.322479 1611 ts_tablet_manager.cc:1939] T 89f95258561a41c19711632fd3a85e90 P df6e68db6f6a4a699e2282f05fb32b5d: Deleting consensus metadata
I20260811 22:15:16.324226 1612 ts_tablet_manager.cc:1918] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:16.324488 1612 log.cc:1196] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/89f95258561a41c19711632fd3a85e90
I20260811 22:15:16.324852 1608 ts_tablet_manager.cc:1918] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:16.324813 774 catalog_manager.cc:4841] TS df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483): tablet 89f95258561a41c19711632fd3a85e90 (table test-table [id=804dc97399ea48fc86ce7c6715bf1c8e]) successfully deleted
I20260811 22:15:16.325060 1608 log.cc:1196] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/89f95258561a41c19711632fd3a85e90
I20260811 22:15:16.325501 1612 ts_tablet_manager.cc:1939] T 89f95258561a41c19711632fd3a85e90 P 3469cb92c95a455c892fe73f075e6ffc: Deleting consensus metadata
I20260811 22:15:16.325639 1608 ts_tablet_manager.cc:1939] T 89f95258561a41c19711632fd3a85e90 P d5168b81aebe4fb89a48ebcf5a930b78: Deleting consensus metadata
I20260811 22:15:16.327423 777 catalog_manager.cc:4841] TS 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805): tablet 89f95258561a41c19711632fd3a85e90 (table test-table [id=804dc97399ea48fc86ce7c6715bf1c8e]) successfully deleted
I20260811 22:15:16.329663 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:16.298609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55306 (local address 127.10.173.129:45097)
0811 22:15:16.298851 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:16.298861 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:16.307060 (+ 8199us) server_negotiation.cc:407] Connection header received
0811 22:15:16.307277 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:16.307286 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:16.307397 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:16.307548 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:16.310178 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.311421 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.313692 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.314433 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.314542 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:16.321341 (+ 6799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:16.321360 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:16.321381 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:16.321438 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:16.325940 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:16.326938 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:16.326951 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:16.326955 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:16.327143 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:16.327679 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:16.327687 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:16.327690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:16.328177 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:16.328407 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:16.328936 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 22:15:16.329176 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:15:16.330345 1086 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:16.429433 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 892
I20260811 22:15:16.429664 777 catalog_manager.cc:4841] TS d5168b81aebe4fb89a48ebcf5a930b78 (127.10.173.129:45097): tablet 89f95258561a41c19711632fd3a85e90 (table test-table [id=804dc97399ea48fc86ce7c6715bf1c8e]) successfully deleted
I20260811 22:15:16.460247 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 1102
Aug 11 22:15:16 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:15:16.480800 1544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:16.461190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41764 (local address 127.10.173.130:32805)
0811 22:15:16.461405 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:16.461411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:16.461434 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:15:16.461503 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:16.461507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:16.461582 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:16.461700 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:16.463105 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.463896 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.465285 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.465691 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.465844 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:16.474709 (+ 8865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:16.474731 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:16.474749 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:16.474793 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:16.477785 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:16.478567 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:16.478575 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:16.478578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:16.478713 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:16.479143 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:16.479147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:16.479149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:16.479552 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:16.479890 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:16.480226 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:15:16.480458 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:15:16.481666 1302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:16.572564 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 1102
I20260811 22:15:16.598498 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 1317
Aug 11 22:15:16 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:15:16.619496 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:16.599379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58220 (local address 127.10.173.131:40483)
0811 22:15:16.599593 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:16.599599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:16.599621 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:15:16.599786 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:16.599792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:16.599873 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:16.599996 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:16.601333 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.602077 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.603477 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:16.603896 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:16.604041 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:16.613511 (+ 9470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:16.613529 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:16.613547 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:16.613583 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:16.616344 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:16.617065 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:16.617073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:16.617077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:16.617293 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:16.617718 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:16.617724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:16.617729 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:16.618099 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:16.618413 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:16.618917 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 22:15:16.619146 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:15:16.620215 1520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:16.711889 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 1317
I20260811 22:15:16.735720 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 752
I20260811 22:15:16.736893 833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:16.849116 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 752
I20260811 22:15:16.883226 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38319
--webserver_interface=127.10.173.190
--webserver_port=34677
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:38319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:17.235781 1620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:17.236063 1620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:17.236235 1620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:17.243441 1620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:17.243564 1620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:17.243646 1620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:17.243752 1620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:17.243825 1620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:17.253629 1620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:38319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=34677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:17.254940 1620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:17.256372 1620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:17.283505 1636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:17.283553 1632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:17.283475 1631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:17.284679 1620 server_base.cc:1034] running on GCE node
I20260811 22:15:17.285696 1620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:17.287501 1620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:17.288774 1620 hybrid_clock.cc:648] HybridClock initialized: now 1786486517288713 us; error 52 us; skew 500 ppm
I20260811 22:15:17.289225 1620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:17 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486517, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:17.293228 1620 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:15:17.295178 1620 webserver.cc:460] Webserver started at http://127.10.173.190:34677/ using document root <none> and password file <none>
I20260811 22:15:17.296054 1620 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:17.296195 1620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:17.302654 1620 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:15:17.306157 1644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:17.307842 1620 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:15:17.308003 1620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "daa57e0e4bf346e29bf305e63136797b"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-68xk"
server_key: "4b1765d492f85902e48b493c7dbb32e5"
server_key_iv: "1c6cecbcd037bb31a9a0e1015ce87e8b"
server_key_version: "encryptionkey@0"
I20260811 22:15:17.309118 1620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:17.337280 1620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:17.342272 1620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:17.342612 1620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:17.363488 1620 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:38319
I20260811 22:15:17.363507 1727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:38319 every 8 connection(s)
I20260811 22:15:17.365131 1620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:17.372558 1731 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:15:17.373283 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 1620
I20260811 22:15:17.374091 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:45097
--local_ip_for_outbound_sockets=127.10.173.129
--tserver_master_addrs=127.10.173.190:38319
--webserver_port=35633
--webserver_interface=127.10.173.129
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:15:17.376236 1731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Bootstrap starting.
I20260811 22:15:17.396440 1731 log.cc:824] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:17.409143 1731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:15:17.409739 1731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Bootstrap complete.
I20260811 22:15:17.419829 1731 raft_consensus.cc:348] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:17.420737 1731 raft_consensus.cc:729] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: daa57e0e4bf346e29bf305e63136797b, State: Initialized, Role: FOLLOWER
I20260811 22:15:17.421355 1731 consensus_queue.cc:260] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:17.421622 1731 raft_consensus.cc:388] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:17.421762 1731 raft_consensus.cc:482] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:17.421953 1731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:15:17.425979 1731 raft_consensus.cc:504] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:17.426667 1731 leader_election.cc:302] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daa57e0e4bf346e29bf305e63136797b; no voters:
I20260811 22:15:17.427352 1731 leader_election.cc:288] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:15:17.427520 1739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:15:17.428880 1739 raft_consensus.cc:686] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 2 LEADER]: Becoming Leader. State: Replica: daa57e0e4bf346e29bf305e63136797b, State: Running, Role: LEADER
I20260811 22:15:17.429450 1739 consensus_queue.cc:237] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:17.429812 1731 sys_catalog.cc:564] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:17.432188 1742 sys_catalog.cc:455] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: SysCatalogTable state changed. Reason: New leader daa57e0e4bf346e29bf305e63136797b. Latest consensus state: current_term: 2 leader_uuid: "daa57e0e4bf346e29bf305e63136797b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } } }
I20260811 22:15:17.432648 1742 sys_catalog.cc:458] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:17.433188 1741 sys_catalog.cc:455] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "daa57e0e4bf346e29bf305e63136797b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } } }
I20260811 22:15:17.433543 1741 sys_catalog.cc:458] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:17.438246 1749 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:17.448400 1749 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:17.450309 1749 catalog_manager.cc:1259] Loaded cluster ID: 22f3e66a9d394596930f325132324e48
I20260811 22:15:17.450520 1749 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:17.457585 1749 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 22:15:17.483042 1749 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 22:15:17.491874 1749 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:17.493233 1749 catalog_manager.cc:5879] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Loaded TSK: 0
I20260811 22:15:17.494561 1749 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:15:17.749400 1736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:17.749727 1736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:17.749887 1736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:17.758111 1736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:17.758237 1736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:17.758458 1736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:15:17.768512 1736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:45097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=35633
--tserver_master_addrs=127.10.173.190:38319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:17.769884 1736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:17.771407 1736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:17.786603 1776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:17.786711 1777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:17.787457 1779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:17.787932 1736 server_base.cc:1034] running on GCE node
I20260811 22:15:17.788640 1736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:17.789628 1736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:17.790886 1736 hybrid_clock.cc:648] HybridClock initialized: now 1786486517790835 us; error 44 us; skew 500 ppm
I20260811 22:15:17.791232 1736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:17 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486517, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:17.794569 1736 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:15:17.796324 1736 webserver.cc:460] Webserver started at http://127.10.173.129:35633/ using document root <none> and password file <none>
I20260811 22:15:17.797040 1736 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:17.797214 1736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:17.803087 1736 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:15:17.806725 1796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:17.808244 1736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:17.808445 1736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "d5168b81aebe4fb89a48ebcf5a930b78"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-68xk"
server_key: "da207de210b7a7344a28c5502a07c665"
server_key_iv: "2724aac1669697d846ad34d3f54c415a"
server_key_version: "encryptionkey@0"
I20260811 22:15:17.809453 1736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:17.826640 1736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:17.831245 1736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:17.831537 1736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:17.832549 1736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:17.833884 1736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:17.834017 1736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:17.834146 1736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:17.834241 1736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:17.869280 1736 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:45097
I20260811 22:15:17.869344 1982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:45097 every 8 connection(s)
I20260811 22:15:17.871093 1736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:17.872805 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 1736
I20260811 22:15:17.873633 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:32805
--local_ip_for_outbound_sockets=127.10.173.130
--tserver_master_addrs=127.10.173.190:38319
--webserver_port=36743
--webserver_interface=127.10.173.130
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:15:17 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486517, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:17.901823 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:17.874784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:54409)
0811 22:15:17.875864 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:17.875928 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:17.879038 (+ 3110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:17.880794 (+ 1756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:17.880802 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:17.881364 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:17.881938 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:17.881956 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:17.884680 (+ 2724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:17.884684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:17.885968 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:17.885982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:17.887696 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:17.887701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:17.887928 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:17.888748 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:17.888772 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:17.891541 (+ 2769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:17.897074 (+ 5533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:17.897082 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:17.897095 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:17.897568 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:17.898213 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:17.898218 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:17.898221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:17.898545 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:17.899674 (+ 1129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:17.899680 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:17.899999 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 22:15:17.900235 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 22:15:17.900666 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":409,"threads_started":1}
I20260811 22:15:17.902576 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:17.874473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:54409 (local address 127.10.173.190:38319)
0811 22:15:17.879030 (+ 4557us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:17.879052 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:17.879113 (+ 61us) server_negotiation.cc:407] Connection header received
0811 22:15:17.879672 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:17.879694 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:17.880202 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:17.880665 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:17.882220 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:17.884481 (+ 2261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:17.886147 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:17.887428 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:17.887651 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:17.891884 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:17.891899 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:17.891923 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:17.891979 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:17.896861 (+ 4882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:17.897731 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:17.897745 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:17.897756 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:17.898002 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:17.898701 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:17.898709 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:17.898716 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:17.899486 (+ 770us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:17.899723 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:17.900577 (+ 854us) server_negotiation.cc:299] Negotiation successful
0811 22:15:17.901086 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4275,"thread_start_us":4134,"threads_started":1}
I20260811 22:15:17.905767 1984 heartbeater.cc:347] Connected to a master server at 127.10.173.190:38319
I20260811 22:15:17.906102 1984 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:17.906962 1984 heartbeater.cc:511] Master 127.10.173.190:38319 requested a full tablet report, sending...
I20260811 22:15:17.909334 1669 ts_manager.cc:194] Registered new tserver with Master: d5168b81aebe4fb89a48ebcf5a930b78 (127.10.173.129:45097)
I20260811 22:15:17.910939 1669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:54409
W20260811 22:15:18.236877 1990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:18.237146 1990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:18.237244 1990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:18.244415 1990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:18.244526 1990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:18.244696 1990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:15:18.254585 1990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:32805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=36743
--tserver_master_addrs=127.10.173.190:38319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:18.255828 1990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:18.257123 1990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:18.271181 2011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:18.271303 2006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:18.271328 2007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:18.272547 1990 server_base.cc:1034] running on GCE node
I20260811 22:15:18.273149 1990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:18.274111 1990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:18.275352 1990 hybrid_clock.cc:648] HybridClock initialized: now 1786486518275315 us; error 55 us; skew 500 ppm
I20260811 22:15:18.275679 1990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:18 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486518, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:18.279345 1990 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:15:18.281003 1990 webserver.cc:460] Webserver started at http://127.10.173.130:36743/ using document root <none> and password file <none>
I20260811 22:15:18.281739 1990 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:18.281873 1990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:18.287837 1990 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:15:18.290757 2023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:18.292181 1990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:15:18.292325 1990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "3469cb92c95a455c892fe73f075e6ffc"
format_stamp: "Formatted at 2026-08-11 22:15:15 on dist-test-slave-68xk"
server_key: "f6ec04255bc5310a7060569cfe6e499d"
server_key_iv: "c831cb69186889439d9cfdb9fa032628"
server_key_version: "encryptionkey@0"
I20260811 22:15:18.293365 1990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:18.314549 1990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:18.319279 1990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:18.319536 1990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:18.320554 1990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:18.322222 1990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:18.322322 1990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:18.322438 1990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:18.322499 1990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 22:15:18.356693 1990 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:32805
I20260811 22:15:18.356753 2196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:32805 every 8 connection(s)
I20260811 22:15:18.358310 1990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:18.360010 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 1990
I20260811 22:15:18.360780 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:40483
--local_ip_for_outbound_sockets=127.10.173.131
--tserver_master_addrs=127.10.173.190:38319
--webserver_port=33199
--webserver_interface=127.10.173.131
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:15:18 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486518, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:18.386399 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.361378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:59051 (local address 127.10.173.190:38319)
0811 22:15:18.361572 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:18.361577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:18.363203 (+ 1626us) server_negotiation.cc:407] Connection header received
0811 22:15:18.365909 (+ 2706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:18.365919 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:18.366053 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:18.366210 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:18.368219 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.369482 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.371820 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.372527 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.372617 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.377574 (+ 4957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:18.377595 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:18.377604 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:18.377657 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:18.382728 (+ 5071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:18.383488 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:18.383498 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:18.383505 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:18.383707 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:18.384234 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:18.384243 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:18.384250 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:18.384679 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:18.384853 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:18.385607 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 22:15:18.385915 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:15:18.387022 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.362058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:59051)
0811 22:15:18.363031 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:18.363088 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:18.365573 (+ 2485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:18.366501 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:18.366513 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:18.367249 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:18.368009 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.368033 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.369684 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.369690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.371633 (+ 1943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.371648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.372960 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.372967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.373252 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.374119 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:18.374152 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:18.377290 (+ 3138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:18.382879 (+ 5589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:18.382886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:18.382899 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:18.383335 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:18.383880 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:18.383884 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:18.383887 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:18.384084 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:18.384824 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:18.384830 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:18.385164 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 22:15:18.385395 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 22:15:18.385798 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":417,"threads_started":1}
I20260811 22:15:18.388546 2197 heartbeater.cc:347] Connected to a master server at 127.10.173.190:38319
I20260811 22:15:18.388870 2197 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:18.389612 2197 heartbeater.cc:511] Master 127.10.173.190:38319 requested a full tablet report, sending...
I20260811 22:15:18.391191 1669 ts_manager.cc:194] Registered new tserver with Master: 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805)
I20260811 22:15:18.391988 1669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:59051
W20260811 22:15:18.713304 2202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:18.713672 2202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:18.713845 2202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:18.721007 2202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:18.721175 2202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:18.721364 2202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:15:18.731457 2202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:40483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=33199
--tserver_master_addrs=127.10.173.190:38319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:18.732843 2202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:18.734222 2202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:18.747869 2213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:18.748030 2212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:18.748039 2217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:18.749187 2202 server_base.cc:1034] running on GCE node
I20260811 22:15:18.749854 2202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:18.750878 2202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:18.752123 2202 hybrid_clock.cc:648] HybridClock initialized: now 1786486518752062 us; error 73 us; skew 500 ppm
I20260811 22:15:18.752470 2202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:18 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486518, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:18.755708 2202 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:15:18.757122 2202 webserver.cc:460] Webserver started at http://127.10.173.131:33199/ using document root <none> and password file <none>
I20260811 22:15:18.757808 2202 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:18.757972 2202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:18.763356 2202 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:15:18.766301 2227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:18.767836 2202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:15:18.768055 2202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "df6e68db6f6a4a699e2282f05fb32b5d"
format_stamp: "Formatted at 2026-08-11 22:15:15 on dist-test-slave-68xk"
server_key: "db5ef7df951946dd0d35078b1de48c86"
server_key_iv: "42917e676a88144b75745b850e8896ea"
server_key_version: "encryptionkey@0"
I20260811 22:15:18.769105 2202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:18.798117 2202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:18.802639 2202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:18.802929 2202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:18.803972 2202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:18.805357 2202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:18.805457 2202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:18.805619 2202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:18.805711 2202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:18.840613 2202 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:40483
I20260811 22:15:18.840708 2399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:40483 every 8 connection(s)
I20260811 22:15:18.842181 2202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:18.844817 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 2202
Aug 11 22:15:18 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486518, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:18.865731 2406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.846345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47112 (local address 127.10.173.190:38319)
0811 22:15:18.849007 (+ 2662us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:18.849011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:18.849025 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:15:18.849067 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:18.849070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:18.849132 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:18.849216 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:18.851458 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.852593 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.854603 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.855155 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.855339 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.859003 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:18.859008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:18.859011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:18.859043 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:18.862814 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:18.863501 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:18.863512 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:18.863519 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:18.863697 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:18.864129 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:18.864133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:18.864135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:18.864438 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:18.864725 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:18.865021 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 22:15:18.865296 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":267,"server-negotiator.queue_time_us":2577,"thread_start_us":2189,"threads_started":1}
I20260811 22:15:18.867256 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.846331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:52699 (local address 127.10.173.190:38319)
0811 22:15:18.846563 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:18.846571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:18.846749 (+ 178us) server_negotiation.cc:407] Connection header received
0811 22:15:18.848737 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:18.848747 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:18.848867 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:18.849019 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:18.851330 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.852592 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.854928 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.855620 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.855710 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.860310 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:18.860317 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:18.860324 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:18.860371 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:18.863621 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:18.864484 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:18.864490 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:18.864495 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:18.864637 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:18.865161 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:18.865165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:18.865170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:18.865486 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:18.865615 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:18.866808 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0811 22:15:18.866963 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"spinlock_wait_cycles":7424}
I20260811 22:15:18.868475 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.845702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:52699)
0811 22:15:18.846579 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:18.846632 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:18.848454 (+ 1822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:18.849305 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:18.849318 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:18.850163 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:18.851127 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.851160 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.852785 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.852790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.854795 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.854808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.856216 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.856227 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.856561 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.857280 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:18.857319 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:18.860056 (+ 2737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:18.863787 (+ 3731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:18.863796 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:18.863813 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:18.864353 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:18.864761 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:18.864765 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:18.864768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:18.865021 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:18.865705 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:18.865715 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:18.866194 (+ 479us) client_negotiation.cc:769] Sending connection context
0811 22:15:18.866586 (+ 392us) client_negotiation.cc:240] Negotiation successful
0811 22:15:18.867162 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":550,"threads_started":1}
I20260811 22:15:18.870085 2400 heartbeater.cc:347] Connected to a master server at 127.10.173.190:38319
I20260811 22:15:18.870473 2400 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:18.871232 2400 heartbeater.cc:511] Master 127.10.173.190:38319 requested a full tablet report, sending...
I20260811 22:15:18.872694 1669 ts_manager.cc:194] Registered new tserver with Master: df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483)
I20260811 22:15:18.873857 1669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:52699
I20260811 22:15:18.881259 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:15:18.888427 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.882285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47116 (local address 127.10.173.190:38319)
0811 22:15:18.882524 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:18.882528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:18.883015 (+ 487us) server_negotiation.cc:407] Connection header received
0811 22:15:18.883174 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:18.883178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:18.883335 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:18.883472 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:18.883884 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.884705 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.886190 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.886699 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.886800 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.887347 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:18.887890 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:18.888016 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:18.888159 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 22:15:18.888231 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:15:18.889348 1669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:15:18.896926 1669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:18.914301 1984 heartbeater.cc:503] Master 127.10.173.190:38319 was elected leader, sending a full tablet report...
I20260811 22:15:18.924315 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.910122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.10.173.129:45097)
0811 22:15:18.913929 (+ 3807us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:18.913939 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:18.913972 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:15:18.914168 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:18.914185 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:18.915308 (+ 1123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:18.916451 (+ 1143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:18.916759 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.918599 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.921278 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.922912 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.923143 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.923362 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:18.923759 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 22:15:18.923948 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3613,"thread_start_us":3344,"threads_started":1}
I20260811 22:15:18.924434 2417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.910016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45097 (local address 127.0.0.1:55310)
0811 22:15:18.912314 (+ 2298us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:18.912364 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:18.912556 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:18.915601 (+ 3045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:18.915609 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:18.915659 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:18.916187 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.916208 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.918752 (+ 2544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.918758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.921099 (+ 2341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.921114 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.923168 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.923175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.923436 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.923503 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 22:15:18.923693 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:15:18.923938 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2045,"mutex_wait_us":112,"spinlock_wait_cycles":5888,"thread_start_us":3784,"threads_started":1}
I20260811 22:15:18.925166 2419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.910768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58234 (local address 127.10.173.131:40483)
0811 22:15:18.913245 (+ 2477us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:18.913256 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:18.915515 (+ 2259us) server_negotiation.cc:407] Connection header received
0811 22:15:18.915863 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:18.915874 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:18.916118 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:18.916304 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:18.917874 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.919369 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.921709 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.923248 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.923467 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.923646 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:18.924279 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 22:15:18.924474 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2212,"thread_start_us":2019,"threads_started":1}
I20260811 22:15:18.925209 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.914200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:32805 (local address 127.0.0.1:41778)
0811 22:15:18.914943 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:18.914964 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:18.915092 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:18.916049 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:18.916054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:18.916085 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:18.916466 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.916476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.918261 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.918269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.920987 (+ 2718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.921034 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.923634 (+ 2600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.923638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.923839 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.924519 (+ 680us) client_negotiation.cc:769] Sending connection context
0811 22:15:18.924620 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 22:15:18.924762 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":686,"thread_start_us":403,"threads_started":1}
I20260811 22:15:18.925333 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.914578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41778 (local address 127.10.173.130:32805)
0811 22:15:18.915189 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:18.915199 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:18.915239 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:15:18.915348 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:18.915356 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:18.915577 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:18.915795 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:18.916630 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.918115 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.921236 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.922958 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.923105 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.924031 (+ 926us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:18.924756 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 22:15:18.924944 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":295,"threads_started":1}
I20260811 22:15:18.926906 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:18.910089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:40483 (local address 127.0.0.1:58234)
0811 22:15:18.915392 (+ 5303us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:18.915416 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:18.915642 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:18.916700 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:18.916707 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:18.916738 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:18.917426 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.917441 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.919533 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.919539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.921468 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:18.921482 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:18.923847 (+ 2365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:18.923854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:18.924058 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:18.924103 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 22:15:18.924209 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:15:18.924348 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3868,"mutex_wait_us":47,"thread_start_us":1812,"threads_started":1}
I20260811 22:15:18.928854 1880 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1f28e1dc9843388c9b6da6e7949b75 (DEFAULT_TABLE table=test-table [id=aab13f443e714d7ca6667da511e0c196]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:18.929279 2096 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1f28e1dc9843388c9b6da6e7949b75 (DEFAULT_TABLE table=test-table [id=aab13f443e714d7ca6667da511e0c196]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:18.930291 1880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1f28e1dc9843388c9b6da6e7949b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:18.930792 2096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1f28e1dc9843388c9b6da6e7949b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:18.930912 2306 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1f28e1dc9843388c9b6da6e7949b75 (DEFAULT_TABLE table=test-table [id=aab13f443e714d7ca6667da511e0c196]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:18.932355 2306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1f28e1dc9843388c9b6da6e7949b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:18.942958 2422 tablet_bootstrap.cc:492] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Bootstrap starting.
I20260811 22:15:18.946529 2422 tablet_bootstrap.cc:654] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:18.947729 2423 tablet_bootstrap.cc:492] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: Bootstrap starting.
I20260811 22:15:18.947850 2422 log.cc:824] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:18.949836 2425 tablet_bootstrap.cc:492] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: Bootstrap starting.
I20260811 22:15:18.951962 2423 tablet_bootstrap.cc:654] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:18.952286 2422 tablet_bootstrap.cc:492] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: No bootstrap required, opened a new log
I20260811 22:15:18.952584 2422 ts_tablet_manager.cc:1397] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 22:15:18.953274 2423 log.cc:824] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:18.955331 2425 tablet_bootstrap.cc:654] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:18.956352 2425 log.cc:824] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:18.956406 2423 tablet_bootstrap.cc:492] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: No bootstrap required, opened a new log
I20260811 22:15:18.956705 2423 ts_tablet_manager.cc:1397] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 22:15:18.959146 2425 tablet_bootstrap.cc:492] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: No bootstrap required, opened a new log
I20260811 22:15:18.959749 2425 ts_tablet_manager.cc:1397] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 22:15:18.963274 2422 raft_consensus.cc:348] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:18.963753 2422 raft_consensus.cc:374] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:18.963865 2422 raft_consensus.cc:729] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5168b81aebe4fb89a48ebcf5a930b78, State: Initialized, Role: FOLLOWER
I20260811 22:15:18.964526 2422 consensus_queue.cc:260] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:18.966367 2422 ts_tablet_manager.cc:1428] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 22:15:18.967252 2423 raft_consensus.cc:348] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:18.967667 2423 raft_consensus.cc:374] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:18.967787 2423 raft_consensus.cc:729] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3469cb92c95a455c892fe73f075e6ffc, State: Initialized, Role: FOLLOWER
I20260811 22:15:18.968394 2425 raft_consensus.cc:348] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:18.968390 2423 consensus_queue.cc:260] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:18.968739 2425 raft_consensus.cc:374] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:18.968860 2425 raft_consensus.cc:729] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df6e68db6f6a4a699e2282f05fb32b5d, State: Initialized, Role: FOLLOWER
I20260811 22:15:18.969512 2425 consensus_queue.cc:260] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:18.969741 2197 heartbeater.cc:503] Master 127.10.173.190:38319 was elected leader, sending a full tablet report...
I20260811 22:15:18.970925 2423 ts_tablet_manager.cc:1428] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260811 22:15:18.971849 2400 heartbeater.cc:503] Master 127.10.173.190:38319 was elected leader, sending a full tablet report...
I20260811 22:15:18.972492 2425 ts_tablet_manager.cc:1428] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260811 22:15:19.080189 2429 raft_consensus.cc:482] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:19.080509 2429 raft_consensus.cc:504] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:19.082297 2429 leader_election.cc:288] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483), 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805)
I20260811 22:15:19.091935 2419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.082845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:59211 (local address 127.10.173.131:40483)
0811 22:15:19.083074 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:19.083083 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:19.083716 (+ 633us) server_negotiation.cc:407] Connection header received
0811 22:15:19.083949 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:19.083954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:19.084021 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:19.084111 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:19.084827 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.086017 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.089350 (+ 3333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.090703 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.090854 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.090928 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:19.091520 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 22:15:19.091660 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:15:19.091935 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.083391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:56023 (local address 127.10.173.130:32805)
0811 22:15:19.083609 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:19.083615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:19.083696 (+ 81us) server_negotiation.cc:407] Connection header received
0811 22:15:19.084279 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:19.084283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:19.084344 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:19.084411 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:19.085076 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.086150 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.089350 (+ 3200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.090678 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.090851 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.090928 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:19.091520 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 22:15:19.091680 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:15:19.092172 2435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.082655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:32805 (local address 127.10.173.129:56023)
0811 22:15:19.083572 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.083596 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:19.083788 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:19.084466 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:19.084472 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:19.084504 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:19.084910 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:19.084920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.086297 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.086304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:19.089141 (+ 2837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:19.089158 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.090983 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.090992 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:19.091272 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.091345 (+ 73us) client_negotiation.cc:769] Sending connection context
0811 22:15:19.091446 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 22:15:19.091667 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"mutex_wait_us":264,"thread_start_us":908,"threads_started":1}
I20260811 22:15:19.092509 2436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.082883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:40483 (local address 127.10.173.129:59211)
0811 22:15:19.083607 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.083626 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:19.083804 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:19.084149 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:19.084153 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:19.084176 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:19.084617 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:19.084636 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.086167 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.086170 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:19.089146 (+ 2976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:19.089158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.090999 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.091005 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:19.091207 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.091341 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 22:15:19.091446 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:15:19.091667 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":305,"thread_start_us":406,"threads_started":1}
I20260811 22:15:19.093406 2129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1f28e1dc9843388c9b6da6e7949b75" candidate_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3469cb92c95a455c892fe73f075e6ffc" is_pre_election: true
I20260811 22:15:19.093595 2335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1f28e1dc9843388c9b6da6e7949b75" candidate_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" is_pre_election: true
W20260811 22:15:19.093912 2401 tablet.cc:2369] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:19.093971 2129 raft_consensus.cc:2454] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5168b81aebe4fb89a48ebcf5a930b78 in term 0.
I20260811 22:15:19.094239 2335 raft_consensus.cc:2454] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5168b81aebe4fb89a48ebcf5a930b78 in term 0.
I20260811 22:15:19.094801 1808 leader_election.cc:302] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3469cb92c95a455c892fe73f075e6ffc, d5168b81aebe4fb89a48ebcf5a930b78; no voters:
I20260811 22:15:19.095255 2429 raft_consensus.cc:2781] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:19.095417 2429 raft_consensus.cc:482] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:19.095533 2429 raft_consensus.cc:3037] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:19.097756 2429 raft_consensus.cc:504] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:19.098402 2429 leader_election.cc:288] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 1 election: Requested vote from peers df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483), 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805)
I20260811 22:15:19.098807 2335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1f28e1dc9843388c9b6da6e7949b75" candidate_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df6e68db6f6a4a699e2282f05fb32b5d"
I20260811 22:15:19.099035 2335 raft_consensus.cc:3037] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:19.099033 2129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1f28e1dc9843388c9b6da6e7949b75" candidate_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3469cb92c95a455c892fe73f075e6ffc"
I20260811 22:15:19.099259 2129 raft_consensus.cc:3037] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:19.101431 2335 raft_consensus.cc:2454] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5168b81aebe4fb89a48ebcf5a930b78 in term 1.
I20260811 22:15:19.101847 1802 leader_election.cc:302] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5168b81aebe4fb89a48ebcf5a930b78, df6e68db6f6a4a699e2282f05fb32b5d; no voters:
I20260811 22:15:19.102011 2129 raft_consensus.cc:2454] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5168b81aebe4fb89a48ebcf5a930b78 in term 1.
I20260811 22:15:19.102191 2429 raft_consensus.cc:2781] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:19.102752 2429 raft_consensus.cc:686] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 LEADER]: Becoming Leader. State: Replica: d5168b81aebe4fb89a48ebcf5a930b78, State: Running, Role: LEADER
I20260811 22:15:19.103221 2429 consensus_queue.cc:237] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:19.107856 1669 catalog_manager.cc:5495] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5168b81aebe4fb89a48ebcf5a930b78 (127.10.173.129). New cstate: current_term: 1 leader_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } health_report { overall_health: UNKNOWN } } }
W20260811 22:15:19.110689 2198 tablet.cc:2369] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:15:19.123472 1986 tablet.cc:2369] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:19.159950 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.154570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.10.173.129:45097)
0811 22:15:19.154816 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:19.154820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:19.154838 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:15:19.155112 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:19.155117 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:19.155216 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:19.155348 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:19.155669 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.156439 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.157818 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.158222 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.158315 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.158853 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:19.159376 (+ 523us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:19.159493 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:19.159567 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 22:15:19.159633 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:15:19.168190 2129 raft_consensus.cc:1261] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc [term 1 FOLLOWER]: Refusing update from remote peer d5168b81aebe4fb89a48ebcf5a930b78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:19.168161 2335 raft_consensus.cc:1261] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d [term 1 FOLLOWER]: Refusing update from remote peer d5168b81aebe4fb89a48ebcf5a930b78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:19.169198 2429 consensus_queue.cc:1035] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:19.169504 2438 consensus_queue.cc:1035] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:19.180816 2442 mvcc.cc:205] Tried to move back new op lower bound from 7317448782502727680 to 7317448782250237952. Current Snapshot: MvccSnapshot[applied={T|T < 7317448782502727680}]
I20260811 22:15:19.186533 2443 mvcc.cc:205] Tried to move back new op lower bound from 7317448782502727680 to 7317448782250237952. Current Snapshot: MvccSnapshot[applied={T|T < 7317448782502727680}]
I20260811 22:15:19.201604 1668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:19.201954 1668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:19.205847 1668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Sending DeleteTablet for 3 replicas of tablet 0a1f28e1dc9843388c9b6da6e7949b75
I20260811 22:15:19.207211 2306 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1f28e1dc9843388c9b6da6e7949b75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:19 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:58234
I20260811 22:15:19.207229 1880 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1f28e1dc9843388c9b6da6e7949b75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:19 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:55310
I20260811 22:15:19.207638 2096 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1f28e1dc9843388c9b6da6e7949b75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:19 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:41778
I20260811 22:15:19.208454 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 1736
I20260811 22:15:19.224717 2457 tablet_replica.cc:330] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: stopping tablet replica
I20260811 22:15:19.225314 2457 raft_consensus.cc:2229] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:19.225977 2457 raft_consensus.cc:2258] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:19.228904 2457 ts_tablet_manager.cc:1905] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:19.236611 2457 ts_tablet_manager.cc:1918] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:19.236811 2457 log.cc:1196] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/0a1f28e1dc9843388c9b6da6e7949b75
I20260811 22:15:19.237430 2457 ts_tablet_manager.cc:1939] T 0a1f28e1dc9843388c9b6da6e7949b75 P 3469cb92c95a455c892fe73f075e6ffc: Deleting consensus metadata
I20260811 22:15:19.239796 1650 catalog_manager.cc:4841] TS 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805): tablet 0a1f28e1dc9843388c9b6da6e7949b75 (table test-table [id=aab13f443e714d7ca6667da511e0c196]) successfully deleted
I20260811 22:15:19.243770 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.209508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.10.173.129:45097)
0811 22:15:19.210249 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:19.210258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:19.210282 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:15:19.210358 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:19.210366 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:19.210493 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:19.210608 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:19.213238 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.214895 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.217784 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.218827 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.218983 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.229246 (+ 10263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:19.229265 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:19.229282 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:19.229342 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:19.233885 (+ 4543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:19.239148 (+ 5263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:19.239157 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:19.239161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:19.239388 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:19.240092 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:19.240097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:19.240100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:19.240740 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:19.241136 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:19.242824 (+ 1688us) server_negotiation.cc:299] Negotiation successful
0811 22:15:19.243187 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579}
I20260811 22:15:19.244362 2456 tablet_replica.cc:330] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: stopping tablet replica
I20260811 22:15:19.244625 1975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:19.244838 2455 tablet_replica.cc:330] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: stopping tablet replica
I20260811 22:15:19.245381 2455 raft_consensus.cc:2229] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:19.246085 2455 raft_consensus.cc:2258] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:19.248174 2455 ts_tablet_manager.cc:1905] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:19.253731 2455 ts_tablet_manager.cc:1918] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:19.253906 2455 log.cc:1196] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/0a1f28e1dc9843388c9b6da6e7949b75
I20260811 22:15:19.254475 2455 ts_tablet_manager.cc:1939] T 0a1f28e1dc9843388c9b6da6e7949b75 P df6e68db6f6a4a699e2282f05fb32b5d: Deleting consensus metadata
I20260811 22:15:19.256129 1645 catalog_manager.cc:4841] TS df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483): tablet 0a1f28e1dc9843388c9b6da6e7949b75 (table test-table [id=aab13f443e714d7ca6667da511e0c196]) successfully deleted
I20260811 22:15:19.245114 2456 raft_consensus.cc:2229] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:19.346735 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 1736
W20260811 22:15:19.373132 1650 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.173.129:45097 (error 108)
I20260811 22:15:19.373415 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 1990
I20260811 22:15:19.376122 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.375525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45097 (local address 127.0.0.1:55348)
0811 22:15:19.375808 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.375929 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:45097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
W20260811 22:15:19.376391 1650 catalog_manager.cc:4584] TS d5168b81aebe4fb89a48ebcf5a930b78 (127.10.173.129:45097): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a1f28e1dc9843388c9b6da6e7949b75: Network error: Client connection negotiation failed: client connection to 127.10.173.129:45097: connect: Connection refused (error 111)
I20260811 22:15:19.390980 2421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.374647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.10.173.130:32805)
0811 22:15:19.374820 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:19.374827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:19.374846 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:15:19.374910 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:19.374916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:19.374993 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:19.375116 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:19.376884 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.378108 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.379683 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.380319 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.380445 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.383438 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:19.383455 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:19.383474 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:19.383520 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:19.387694 (+ 4174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:19.388572 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:19.388582 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:19.388586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:19.388756 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:19.389300 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:19.389306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:19.389309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:19.389864 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:19.390109 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:19.390528 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 22:15:19.390731 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:15:19.391678 2189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:19.430455 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.429869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45097 (local address 127.0.0.1:55356)
0811 22:15:19.430095 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.430233 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:45097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 22:15:19.431672 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.431243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45097 (local address 127.0.0.1:55358)
0811 22:15:19.431419 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.431526 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:45097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:15:19.483873 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 1990
I20260811 22:15:19.492846 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.492331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45097 (local address 127.0.0.1:55368)
0811 22:15:19.492565 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.492665 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:45097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 22:15:19.494026 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.493644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45097 (local address 127.0.0.1:55376)
0811 22:15:19.493814 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.493900 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:45097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 22:15:19.510339 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 2202
I20260811 22:15:19.525439 2419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.511255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58250 (local address 127.10.173.131:40483)
0811 22:15:19.511456 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:19.511462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:19.511580 (+ 118us) server_negotiation.cc:407] Connection header received
0811 22:15:19.511805 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:19.511811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:19.511894 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:19.512005 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:19.513625 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.514482 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.516017 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:19.516431 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:19.516576 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:19.519421 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:19.519437 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:19.519455 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:19.519494 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:19.522314 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:19.523080 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:19.523111 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:19.523115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:19.523324 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:19.523741 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:19.523748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:19.523753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:19.524142 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:19.524490 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:19.524816 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 22:15:19.525061 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:15:19.526149 2392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:19.574761 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.574091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45097 (local address 127.0.0.1:55378)
0811 22:15:19.574369 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.574523 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:45097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 22:15:19.576026 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.575560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45097 (local address 127.0.0.1:55390)
0811 22:15:19.575792 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.575885 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:45097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 22:15:19.617286 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 2202
I20260811 22:15:19.645802 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 1620
I20260811 22:15:19.647130 1722 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:19.762953 2415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:19.762242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45097 (local address 127.0.0.1:55394)
0811 22:15:19.762588 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:19.762722 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:45097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260811 22:15:19.762972 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 1620
I20260811 22:15:19.791579 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38319
--webserver_interface=127.10.173.190
--webserver_port=34677
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:38319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:20.143234 2468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:20.143507 2468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:20.143683 2468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:20.150933 2468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:20.151034 2468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:20.151113 2468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:20.151198 2468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:20.151280 2468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:20.161211 2468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:38319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=34677
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:20.162631 2468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:20.164032 2468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:20.176401 2477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:20.176362 2482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:20.176401 2478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:20.177556 2468 server_base.cc:1034] running on GCE node
I20260811 22:15:20.178363 2468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:20.179920 2468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:20.181178 2468 hybrid_clock.cc:648] HybridClock initialized: now 1786486520181100 us; error 64 us; skew 500 ppm
I20260811 22:15:20.181586 2468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:20 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486520, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:20.184950 2468 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:15:20.186707 2468 webserver.cc:460] Webserver started at http://127.10.173.190:34677/ using document root <none> and password file <none>
I20260811 22:15:20.187448 2468 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:20.187590 2468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:20.193249 2468 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 22:15:20.196250 2492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:20.197697 2468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:15:20.197842 2468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "daa57e0e4bf346e29bf305e63136797b"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-68xk"
server_key: "4b1765d492f85902e48b493c7dbb32e5"
server_key_iv: "1c6cecbcd037bb31a9a0e1015ce87e8b"
server_key_version: "encryptionkey@0"
I20260811 22:15:20.198843 2468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:20.232496 2468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:20.237028 2468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:20.237286 2468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:20.257536 2468 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:38319
I20260811 22:15:20.257556 2563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:38319 every 8 connection(s)
I20260811 22:15:20.259205 2468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:20.261569 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 2468
I20260811 22:15:20.262530 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:45097
--local_ip_for_outbound_sockets=127.10.173.129
--tserver_master_addrs=127.10.173.190:38319
--webserver_port=35633
--webserver_interface=127.10.173.129
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:15:20.266372 2566 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:15:20.269603 2566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Bootstrap starting.
I20260811 22:15:20.290123 2566 log.cc:824] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:20.307631 2566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:15:20.308252 2566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b: Bootstrap complete.
I20260811 22:15:20.317660 2566 raft_consensus.cc:348] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:20.318717 2566 raft_consensus.cc:729] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: daa57e0e4bf346e29bf305e63136797b, State: Initialized, Role: FOLLOWER
I20260811 22:15:20.319370 2566 consensus_queue.cc:260] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:20.319638 2566 raft_consensus.cc:388] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:20.319788 2566 raft_consensus.cc:482] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:20.319993 2566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:15:20.324018 2566 raft_consensus.cc:504] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:20.324543 2566 leader_election.cc:302] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daa57e0e4bf346e29bf305e63136797b; no voters:
I20260811 22:15:20.325292 2566 leader_election.cc:288] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:15:20.325448 2571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:15:20.326833 2571 raft_consensus.cc:686] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [term 3 LEADER]: Becoming Leader. State: Replica: daa57e0e4bf346e29bf305e63136797b, State: Running, Role: LEADER
I20260811 22:15:20.327476 2571 consensus_queue.cc:237] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } }
I20260811 22:15:20.328284 2566 sys_catalog.cc:564] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:20.330112 2573 sys_catalog.cc:455] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "daa57e0e4bf346e29bf305e63136797b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } } }
I20260811 22:15:20.330616 2573 sys_catalog.cc:458] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:20.335325 2574 sys_catalog.cc:455] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: SysCatalogTable state changed. Reason: New leader daa57e0e4bf346e29bf305e63136797b. Latest consensus state: current_term: 3 leader_uuid: "daa57e0e4bf346e29bf305e63136797b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa57e0e4bf346e29bf305e63136797b" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38319 } } }
I20260811 22:15:20.335668 2574 sys_catalog.cc:458] T 00000000000000000000000000000000 P daa57e0e4bf346e29bf305e63136797b [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:20.338456 2579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:20.348806 2579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:20.350346 2579 catalog_manager.cc:1259] Loaded cluster ID: 22f3e66a9d394596930f325132324e48
I20260811 22:15:20.350520 2579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 22:15:20.352871 2579 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd5757029ad google::LogMessage::Fail() at ??:0
@ 0x7fd57570588a google::LogMessage::SendToLog() at ??:0
@ 0x7fd5757024de google::LogMessage::Flush() at ??:0
@ 0x7fd575703a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd58b135512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd58b132976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd58b1ba610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd58b1ba439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd57834e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd57835191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd578351799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd57832ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd57b6066db start_thread at ??:0
@ 0x7fd57459c61f clone at ??:0
W20260811 22:15:20.657035 2568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:20.657387 2568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:20.657589 2568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:20.665021 2568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:20.665205 2568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:20.665397 2568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:15:20.675510 2568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:45097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=35633
--tserver_master_addrs=127.10.173.190:38319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:20.676848 2568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:20.678221 2568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:20.692270 2631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:20.692204 2634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:20.692373 2632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:20.693868 2568 server_base.cc:1034] running on GCE node
I20260811 22:15:20.694710 2568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:20.695823 2568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:20.697095 2568 hybrid_clock.cc:648] HybridClock initialized: now 1786486520696994 us; error 83 us; skew 500 ppm
I20260811 22:15:20.697454 2568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:20 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486520, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:20.701278 2568 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:15:20.703076 2568 webserver.cc:460] Webserver started at http://127.10.173.129:35633/ using document root <none> and password file <none>
I20260811 22:15:20.703862 2568 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:20.704013 2568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:20.709962 2568 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:15:20.713397 2647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:20.714942 2568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:15:20.715137 2568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "d5168b81aebe4fb89a48ebcf5a930b78"
format_stamp: "Formatted at 2026-08-11 22:15:14 on dist-test-slave-68xk"
server_key: "da207de210b7a7344a28c5502a07c665"
server_key_iv: "2724aac1669697d846ad34d3f54c415a"
server_key_version: "encryptionkey@0"
I20260811 22:15:20.716120 2568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:20.746556 2568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:20.750638 2568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:20.750921 2568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:20.751940 2568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:20.753818 2655 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 22:15:20.759763 2568 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:15:20.759924 2568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 22:15:20.760076 2568 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:15:20.763376 2568 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:15:20.763487 2568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:20.763887 2655 tablet_bootstrap.cc:492] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Bootstrap starting.
I20260811 22:15:20.792379 2655 log.cc:824] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:20.806321 2568 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:45097
I20260811 22:15:20.806360 2815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:45097 every 8 connection(s)
I20260811 22:15:20.806720 2655 tablet_bootstrap.cc:492] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:15:20.807278 2655 tablet_bootstrap.cc:492] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Bootstrap complete.
I20260811 22:15:20.807786 2568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:20.808914 2655 ts_tablet_manager.cc:1397] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Time spent bootstrapping tablet: real 0.045s user 0.026s sys 0.016s
I20260811 22:15:20.813601 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.810809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:51069)
0811 22:15:20.811812 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.812151 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":431,"threads_started":1}
W20260811 22:15:20.814396 2816 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:38319 (0 consecutive failures): Network error: Failed to ping master at 127.10.173.190:38319: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
I20260811 22:15:20.814980 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 2568
I20260811 22:15:20.815388 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.814988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:36423)
0811 22:15:20.815162 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.815262 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:15:20.815831 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:32805
--local_ip_for_outbound_sockets=127.10.173.130
--tserver_master_addrs=127.10.173.190:38319
--webserver_port=36743
--webserver_interface=127.10.173.130
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:15:20.816465 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.816045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:53625)
0811 22:15:20.816236 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.816315 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260811 22:15:20.816720 2816 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:15:20.819689 2655 raft_consensus.cc:348] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:20.821154 2655 raft_consensus.cc:729] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5168b81aebe4fb89a48ebcf5a930b78, State: Initialized, Role: FOLLOWER
I20260811 22:15:20.821908 2655 consensus_queue.cc:260] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:20.823686 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.823331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:54963)
0811 22:15:20.823489 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.823569 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 22:15:20.823825 2655 ts_tablet_manager.cc:1428] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260811 22:15:20.824738 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:20.824420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:36311)
0811 22:15:20.824551 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:20.824617 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:15:21.179137 2825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:21.179446 2825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:21.179618 2825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:21.186950 2825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:21.187068 2825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:21.187271 2825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:15:21.197165 2825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:32805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=36743
--tserver_master_addrs=127.10.173.190:38319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:21.198546 2825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:21.199944 2825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:21.213658 2832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:21.213706 2835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:21.213778 2840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:21.214428 2825 server_base.cc:1034] running on GCE node
I20260811 22:15:21.215232 2825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:21.216284 2825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:21.217533 2825 hybrid_clock.cc:648] HybridClock initialized: now 1786486521217482 us; error 60 us; skew 500 ppm
I20260811 22:15:21.217870 2825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:21 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486521, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:21.221428 2825 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:15:21.223097 2825 webserver.cc:460] Webserver started at http://127.10.173.130:36743/ using document root <none> and password file <none>
I20260811 22:15:21.223805 2825 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:21.223944 2825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:21.229372 2825 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:15:21.232357 2848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:21.233675 2825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:15:21.233863 2825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "3469cb92c95a455c892fe73f075e6ffc"
format_stamp: "Formatted at 2026-08-11 22:15:15 on dist-test-slave-68xk"
server_key: "f6ec04255bc5310a7060569cfe6e499d"
server_key_iv: "c831cb69186889439d9cfdb9fa032628"
server_key_version: "encryptionkey@0"
I20260811 22:15:21.234906 2825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:21.265729 2825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:21.270016 2825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:21.270320 2825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:21.271427 2825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:21.272805 2825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:21.272903 2825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:21.273056 2825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:21.273159 2825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:21.308090 2825 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:32805
I20260811 22:15:21.308185 3019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:32805 every 8 connection(s)
I20260811 22:15:21.309643 2825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:21.314766 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:21.312799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:49281)
0811 22:15:21.313562 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:21.313809 (+ 247us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":331,"threads_started":1}
W20260811 22:15:21.315449 3021 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:38319 (0 consecutive failures): Network error: Failed to ping master at 127.10.173.190:38319: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
I20260811 22:15:21.316267 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:21.315937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:60169)
0811 22:15:21.316084 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:21.316154 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 22:15:21.317142 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:21.316838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:35301)
0811 22:15:21.316961 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:21.317033 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:15:21.317451 3021 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:15:21.317863 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 2825
I20260811 22:15:21.318671 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:40483
--local_ip_for_outbound_sockets=127.10.173.131
--tserver_master_addrs=127.10.173.190:38319
--webserver_port=33199
--webserver_interface=127.10.173.131
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:45833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:21.664976 3025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:21.665282 3025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:21.665390 3025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:21.673170 3025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:21.673280 3025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:21.673394 3025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:15:21.682866 3025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:45833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:40483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=33199
--tserver_master_addrs=127.10.173.190:38319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:21.684080 3025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:21.685392 3025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:21.699018 3037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:21.698984 3036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:21.698927 3039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:21.699827 3025 server_base.cc:1034] running on GCE node
I20260811 22:15:21.700492 3025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:21.701413 3025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:21.702675 3025 hybrid_clock.cc:648] HybridClock initialized: now 1786486521702619 us; error 63 us; skew 500 ppm
I20260811 22:15:21.703022 3025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:21 dist-test-slave-68xk krb5kdc[726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486521, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:21.711310 3025 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:15:21.712950 3025 webserver.cc:460] Webserver started at http://127.10.173.131:33199/ using document root <none> and password file <none>
I20260811 22:15:21.713714 3025 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:21.713887 3025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:21.719568 3025 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:15:21.722532 3047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:21.723907 3025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:15:21.724107 3025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "df6e68db6f6a4a699e2282f05fb32b5d"
format_stamp: "Formatted at 2026-08-11 22:15:15 on dist-test-slave-68xk"
server_key: "db5ef7df951946dd0d35078b1de48c86"
server_key_iv: "42917e676a88144b75745b850e8896ea"
server_key_version: "encryptionkey@0"
I20260811 22:15:21.725116 3025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:21.752254 3025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:21.756716 3025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:21.757009 3025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:21.758093 3025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:21.759574 3025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:21.759701 3025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:21.759845 3025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:21.759936 3025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:21.794770 3025 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:40483
I20260811 22:15:21.794862 3216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:40483 every 8 connection(s)
I20260811 22:15:21.796424 3025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:21.798933 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 3025
I20260811 22:15:21.801594 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:21.799555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:58885)
0811 22:15:21.800431 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:21.800716 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":355,"threads_started":1}
W20260811 22:15:21.802274 3217 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:38319 (0 consecutive failures): Network error: Failed to ping master at 127.10.173.190:38319: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
I20260811 22:15:21.803143 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:21.802830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:56869)
0811 22:15:21.802966 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:21.803032 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 22:15:21.803973 3222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:21.803675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:49649)
0811 22:15:21.803800 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:21.803872 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:15:21.804251 3217 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:15:21.826346 3226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:21.825537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:35955)
0811 22:15:21.826005 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:21.826136 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":231,"threads_started":1}
I20260811 22:15:22.020176 3227 raft_consensus.cc:482] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:22.020547 3227 raft_consensus.cc:504] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:22.022400 3227 leader_election.cc:288] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483), 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805)
Aug 11 22:15:22 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486520, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
Aug 11 22:15:22 dist-test-slave-68xk krb5kdc[726](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486520, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:15:22.051393 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:22.023150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:44829 (local address 127.10.173.130:32805)
0811 22:15:22.024684 (+ 1534us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:22.024702 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:22.025310 (+ 608us) server_negotiation.cc:407] Connection header received
0811 22:15:22.027555 (+ 2245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:22.027569 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:22.028083 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:22.028463 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:22.031290 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.033256 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.036258 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.037236 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.037427 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:22.042758 (+ 5331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:22.042770 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:22.042791 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:22.042844 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:22.047012 (+ 4168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:22.048182 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:22.048191 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:22.048200 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:22.048434 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:22.048927 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:22.048932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:22.048936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:22.049478 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:22.049688 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:22.050580 (+ 892us) server_negotiation.cc:299] Negotiation successful
0811 22:15:22.050976 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1343,"thread_start_us":1222,"threads_started":1}
I20260811 22:15:22.051412 3228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:22.022904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:32805 (local address 127.10.173.129:44829)
0811 22:15:22.024984 (+ 2080us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:22.025004 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:22.025153 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:22.028658 (+ 3505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:22.028670 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:22.029795 (+ 1125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:22.031038 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:22.031067 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.033468 (+ 2401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.033474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:22.035920 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:22.035937 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.037499 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.037508 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:22.037888 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:22.039042 (+ 1154us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:22.039083 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:22.042372 (+ 3289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:22.047203 (+ 4831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:22.047213 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:22.047230 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:22.047998 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:22.048567 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:22.048573 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:22.048577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:22.048796 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:22.049632 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:22.049641 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:22.049989 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 22:15:22.050281 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 22:15:22.050892 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1939,"mutex_wait_us":469,"spinlock_wait_cycles":34944,"thread_start_us":2185,"threads_started":1}
I20260811 22:15:22.051429 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:22.023047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:37637 (local address 127.10.173.131:40483)
0811 22:15:22.025298 (+ 2251us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:22.025312 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:22.025349 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:15:22.027585 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:22.027593 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:22.028083 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:22.028463 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:22.031290 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.033224 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.036122 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.037228 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.037427 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:22.042925 (+ 5498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:22.042935 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:22.042950 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:22.043010 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:22.047010 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:22.048435 (+ 1425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:22.048442 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:22.048448 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:22.048626 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:22.049052 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:22.049058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:22.049061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:22.049558 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:22.049773 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:22.050593 (+ 820us) server_negotiation.cc:299] Negotiation successful
0811 22:15:22.050990 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2035,"thread_start_us":1833,"threads_started":1}
I20260811 22:15:22.051517 3226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:22.022776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:40483 (local address 127.10.173.129:37637)
0811 22:15:22.023121 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:22.023170 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:22.023808 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:22.028658 (+ 4850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:22.028670 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:22.029795 (+ 1125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:22.031038 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:22.031067 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.033470 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.033474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:22.035920 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:22.035937 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:22.037513 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:22.037519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:22.037888 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:22.039043 (+ 1155us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:22.039075 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:22.042643 (+ 3568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:22.047516 (+ 4873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:22.047521 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:22.047527 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:22.048280 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:22.048765 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:22.048770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:22.048782 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:22.048927 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:22.049695 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:22.049701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:22.049988 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 22:15:22.050287 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 22:15:22.051060 (+ 773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":705,"spinlock_wait_cycles":29568}
W20260811 22:15:22.054791 2652 leader_election.cc:341] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805): Not found: Tablet not found: 0a1f28e1dc9843388c9b6da6e7949b75
W20260811 22:15:22.055070 2648 leader_election.cc:341] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483): Not found: Tablet not found: 0a1f28e1dc9843388c9b6da6e7949b75
I20260811 22:15:22.055199 2648 leader_election.cc:302] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5168b81aebe4fb89a48ebcf5a930b78; no voters: 3469cb92c95a455c892fe73f075e6ffc, df6e68db6f6a4a699e2282f05fb32b5d
I20260811 22:15:22.055574 3227 raft_consensus.cc:2726] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:15:22.318991 3232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:22.318049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:37795)
0811 22:15:22.318601 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:22.318751 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":228,"threads_started":1}
I20260811 22:15:22.805701 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:22.804818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:46925)
0811 22:15:22.805283 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:22.805421 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":217,"threads_started":1}
I20260811 22:15:22.828236 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:22.827296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:54417)
0811 22:15:22.827858 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:22.827992 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":289,"threads_started":1}
I20260811 22:15:23.320778 3235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:23.319985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:55351)
0811 22:15:23.320467 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:23.320558 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":249,"threads_started":1}
I20260811 22:15:23.807467 3236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:23.806726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:44009)
0811 22:15:23.807202 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:23.807293 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":253,"threads_started":1}
I20260811 22:15:23.830209 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:23.829267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:44217)
0811 22:15:23.829820 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:23.829945 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":248,"threads_started":1}
I20260811 22:15:23.938743 3239 raft_consensus.cc:482] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:23.938993 3239 raft_consensus.cc:504] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:23.939777 3239 leader_election.cc:288] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483), 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805)
W20260811 22:15:23.940783 2652 leader_election.cc:341] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805): Not found: Tablet not found: 0a1f28e1dc9843388c9b6da6e7949b75
W20260811 22:15:23.941030 2648 leader_election.cc:341] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483): Not found: Tablet not found: 0a1f28e1dc9843388c9b6da6e7949b75
I20260811 22:15:23.941143 2648 leader_election.cc:302] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5168b81aebe4fb89a48ebcf5a930b78; no voters: 3469cb92c95a455c892fe73f075e6ffc, df6e68db6f6a4a699e2282f05fb32b5d
I20260811 22:15:23.941365 3239 raft_consensus.cc:2726] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:15:24.322818 3241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:24.321808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:40595)
0811 22:15:24.322488 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:24.322585 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":344,"threads_started":1}
I20260811 22:15:24.809168 3242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:24.808453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:54461)
0811 22:15:24.808927 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:24.809017 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":268,"threads_started":1}
I20260811 22:15:24.831935 3243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:24.831202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:53933)
0811 22:15:24.831675 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:24.831776 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":246,"threads_started":1}
I20260811 22:15:25.324750 3247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:25.323828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:37803)
0811 22:15:25.324432 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:25.324530 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":340,"threads_started":1}
I20260811 22:15:25.810891 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:25.810076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:41193)
0811 22:15:25.810616 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:25.810716 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":235,"threads_started":1}
I20260811 22:15:25.833914 3249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:25.833155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:60557)
0811 22:15:25.833639 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:25.833727 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":254,"threads_started":1}
I20260811 22:15:26.326826 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:26.325839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:42757)
0811 22:15:26.326534 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:26.326653 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":363,"threads_started":1}
I20260811 22:15:26.432042 3252 raft_consensus.cc:482] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:26.432313 3252 raft_consensus.cc:504] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:26.433082 3252 leader_election.cc:288] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483), 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805)
W20260811 22:15:26.434088 2648 leader_election.cc:341] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483): Not found: Tablet not found: 0a1f28e1dc9843388c9b6da6e7949b75
W20260811 22:15:26.434265 2652 leader_election.cc:341] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805): Not found: Tablet not found: 0a1f28e1dc9843388c9b6da6e7949b75
I20260811 22:15:26.434443 2652 leader_election.cc:302] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5168b81aebe4fb89a48ebcf5a930b78; no voters: 3469cb92c95a455c892fe73f075e6ffc, df6e68db6f6a4a699e2282f05fb32b5d
I20260811 22:15:26.434765 3252 raft_consensus.cc:2726] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:15:26.812547 3254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:26.811841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:57127)
0811 22:15:26.812317 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:26.812404 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":274,"threads_started":1}
I20260811 22:15:26.835743 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:26.834911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:34477)
0811 22:15:26.835429 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:26.835534 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":271,"threads_started":1}
I20260811 22:15:27.328635 3261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:27.327760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:60583)
0811 22:15:27.328333 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:27.328420 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":267,"threads_started":1}
I20260811 22:15:27.814353 3263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:27.813525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:48417)
0811 22:15:27.814066 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:27.814153 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":287,"threads_started":1}
I20260811 22:15:27.837486 3264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:27.836692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:53839)
0811 22:15:27.837214 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:27.837297 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":272,"threads_started":1}
I20260811 22:15:28.331074 3265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:28.329900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:42829)
0811 22:15:28.330611 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:28.330726 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":323,"threads_started":1}
I20260811 22:15:28.816555 3266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:28.815678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:39147)
0811 22:15:28.816255 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:28.816344 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":248,"threads_started":1}
I20260811 22:15:28.839252 3267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:28.838391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:37607)
0811 22:15:28.838981 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:28.839069 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":312,"threads_started":1}
I20260811 22:15:29.332839 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:29.332071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:41811)
0811 22:15:29.332551 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:29.332639 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":245,"threads_started":1}
I20260811 22:15:29.652318 3269 raft_consensus.cc:482] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:29.652532 3269 raft_consensus.cc:504] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:29.653174 3269 leader_election.cc:288] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483), 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805)
W20260811 22:15:29.654345 2652 leader_election.cc:341] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805): Not found: Tablet not found: 0a1f28e1dc9843388c9b6da6e7949b75
W20260811 22:15:29.654596 2648 leader_election.cc:341] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483): Not found: Tablet not found: 0a1f28e1dc9843388c9b6da6e7949b75
I20260811 22:15:29.654791 2648 leader_election.cc:302] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5168b81aebe4fb89a48ebcf5a930b78; no voters: 3469cb92c95a455c892fe73f075e6ffc, df6e68db6f6a4a699e2282f05fb32b5d
I20260811 22:15:29.655076 3269 raft_consensus.cc:2726] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:15:29.818320 3270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:29.817612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:35997)
0811 22:15:29.818041 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:29.818132 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":233,"threads_started":1}
I20260811 22:15:29.841099 3271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:29.840231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:39759)
0811 22:15:29.840751 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:29.840875 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":263,"threads_started":1}
I20260811 22:15:30.334834 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:30.333823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:47871)
0811 22:15:30.334478 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:30.334609 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":306,"threads_started":1}
I20260811 22:15:30.820197 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:30.819281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:40491)
0811 22:15:30.819891 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:30.819998 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":323,"threads_started":1}
I20260811 22:15:30.843101 3276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:30.842293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:57907)
0811 22:15:30.842846 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:30.842939 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":324,"threads_started":1}
I20260811 22:15:31.336690 3278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:31.335953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:56361)
0811 22:15:31.336417 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:31.336501 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":254,"threads_started":1}
I20260811 22:15:31.822047 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:31.821195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:51729)
0811 22:15:31.821675 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:31.821861 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":236,"threads_started":1}
I20260811 22:15:31.844767 3280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:31.844027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:36455)
0811 22:15:31.844462 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:31.844549 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":241,"threads_started":1}
I20260811 22:15:32.338973 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:32.337879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:34279)
0811 22:15:32.338643 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:32.338783 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":376,"threads_started":1}
I20260811 22:15:32.824028 3282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:32.823230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:46795)
0811 22:15:32.823775 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:32.823868 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":262,"threads_started":1}
I20260811 22:15:32.846899 3283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:32.845954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:48433)
0811 22:15:32.846611 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:32.846706 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":371,"threads_started":1}
I20260811 22:15:33.340807 3284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:33.340018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:37463)
0811 22:15:33.340515 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:33.340605 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":263,"threads_started":1}
I20260811 22:15:33.825848 3286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:33.825082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:47281)
0811 22:15:33.825590 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:33.825673 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":252,"threads_started":1}
I20260811 22:15:33.848985 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:33.847986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:40177)
0811 22:15:33.848703 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:33.848794 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":438,"threads_started":1}
I20260811 22:15:34.269302 3288 raft_consensus.cc:482] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:34.269574 3288 raft_consensus.cc:504] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5168b81aebe4fb89a48ebcf5a930b78" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45097 } } peers { permanent_uuid: "df6e68db6f6a4a699e2282f05fb32b5d" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40483 } } peers { permanent_uuid: "3469cb92c95a455c892fe73f075e6ffc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 32805 } }
I20260811 22:15:34.270349 3288 leader_election.cc:288] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483), 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805)
W20260811 22:15:34.271430 2648 leader_election.cc:341] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer df6e68db6f6a4a699e2282f05fb32b5d (127.10.173.131:40483): Not found: Tablet not found: 0a1f28e1dc9843388c9b6da6e7949b75
W20260811 22:15:34.271647 2652 leader_election.cc:341] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3469cb92c95a455c892fe73f075e6ffc (127.10.173.130:32805): Not found: Tablet not found: 0a1f28e1dc9843388c9b6da6e7949b75
I20260811 22:15:34.271868 2652 leader_election.cc:302] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d5168b81aebe4fb89a48ebcf5a930b78; no voters: 3469cb92c95a455c892fe73f075e6ffc, df6e68db6f6a4a699e2282f05fb32b5d
I20260811 22:15:34.272181 3288 raft_consensus.cc:2726] T 0a1f28e1dc9843388c9b6da6e7949b75 P d5168b81aebe4fb89a48ebcf5a930b78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:15:34.342720 3289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:34.341822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:50589)
0811 22:15:34.342347 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:34.342523 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":263,"threads_started":1}
I20260811 22:15:34.827685 3291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:34.826884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:37207)
0811 22:15:34.827403 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:34.827507 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":255,"threads_started":1}
I20260811 22:15:34.850977 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:34.850014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:52563)
0811 22:15:34.850694 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:34.850781 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":404,"threads_started":1}
I20260811 22:15:35.344494 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:35.343701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:43539)
0811 22:15:35.344185 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:35.344268 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":264,"threads_started":1}
I20260811 22:15:35.829653 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:35.828804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:48977)
0811 22:15:35.829304 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:35.829449 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":239,"threads_started":1}
I20260811 22:15:35.853096 3301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:35.852215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:41435)
0811 22:15:35.852810 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:35.852902 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":285,"threads_started":1}
I20260811 22:15:36.346611 3302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.345795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.130:59391)
0811 22:15:36.346269 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:36.346360 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":240,"threads_started":1}
I20260811 22:15:36.811650 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 2568
I20260811 22:15:36.832060 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.831082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.131:44109)
0811 22:15:36.831634 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:36.831834 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":322,"threads_started":1}
I20260811 22:15:36.833433 3303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.813402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40436 (local address 127.10.173.129:45097)
0811 22:15:36.814024 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:36.814032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:36.814080 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:15:36.814193 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:36.814206 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:36.814569 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:36.814754 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:36.816575 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.817602 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.819011 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.819657 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.819787 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:36.826620 (+ 6833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:36.826636 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:36.826653 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:36.826708 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:36.829985 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:36.831038 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:36.831045 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:36.831048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:36.831200 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:36.831744 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:36.831750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:36.831754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:36.832308 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:36.832614 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:36.832919 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 22:15:36.833145 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":264,"threads_started":1}
I20260811 22:15:36.834820 2811 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:36.929678 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.928530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38319 (local address 127.10.173.129:49621)
0811 22:15:36.929167 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:36.929327 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:38319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":270,"threads_started":1}
I20260811 22:15:36.929677 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 2568
I20260811 22:15:36.960250 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 2825
I20260811 22:15:36.975811 3308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:36.961188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59418 (local address 127.10.173.130:32805)
0811 22:15:36.961741 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:36.961745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:36.961763 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:15:36.961823 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:36.961826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:36.961892 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:36.962037 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:36.963782 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.964586 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.965947 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:36.966488 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:36.966629 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:36.969600 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:36.969606 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:36.969610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:36.969645 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:36.972734 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:36.973489 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:36.973497 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:36.973501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:36.973642 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:36.974086 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:36.974091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:36.974095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:36.974405 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:36.974784 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:36.975128 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 22:15:36.975415 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":271,"threads_started":1}
I20260811 22:15:36.976641 3014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:37.075276 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 2825
I20260811 22:15:37.099344 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 3025
I20260811 22:15:37.114132 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:37.100208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41974 (local address 127.10.173.131:40483)
0811 22:15:37.100855 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:37.100861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:37.100879 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:15:37.100936 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:37.100940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:37.101010 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:37.101106 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:37.102740 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:37.103582 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:37.104954 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:37.105441 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:37.105540 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:37.108629 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:37.108634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:37.108637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:37.108674 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:37.111679 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:37.112414 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:37.112418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:37.112420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:37.112565 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:37.112919 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:37.112922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:37.112924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:37.113170 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:37.113406 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:37.113645 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 22:15:37.113824 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":353,"threads_started":1}
I20260811 22:15:37.114948 3210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:37.203197 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 3025
2026-08-11T22:15:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25339 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:15:37 dist-test-slave-68xk krb5kdc[3314](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:15:37 dist-test-slave-68xk krb5kdc[3314](info): set up 2 sockets
Aug 11 22:15:37 dist-test-slave-68xk krb5kdc[3314](info): commencing operation
krb5kdc: starting...
W20260811 22:15:38.551836 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:15:38 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:15:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:15:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:15:38.846891 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:42607
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:42607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:39.207854 3334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:39.208141 3334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:39.208314 3334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:39.215741 3334 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:15:39.215842 3334 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:15:39.215924 3334 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:15:39.216003 3334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:15:39.216097 3334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:39.216233 3334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:39.216307 3334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:15:39.216395 3334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:15:39.226593 3334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:42607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:42607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:39.227970 3334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:39.229465 3334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:39.241813 3343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.241891 3342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.241938 3345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:39.242878 3334 server_base.cc:1034] running on GCE node
I20260811 22:15:39.243664 3334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:39.245076 3334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:39.246363 3334 hybrid_clock.cc:648] HybridClock initialized: now 1786486539246285 us; error 32 us; skew 500 ppm
I20260811 22:15:39.247037 3334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:39 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486539, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:39.250942 3334 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:15:39.252800 3334 webserver.cc:460] Webserver started at http://127.10.173.190:36385/ using document root <none> and password file <none>
I20260811 22:15:39.253616 3334 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:39.253718 3334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:39.254024 3334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:39.257463 3334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "9069c916037c45fbb1be6e67ddcadbed"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-68xk"
server_key: "10b6c29efe77ea585c4242ffcf58bba6"
server_key_iv: "4ae2f1e31f404f83b5cf322cf7d87a85"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.258294 3334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "9069c916037c45fbb1be6e67ddcadbed"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-68xk"
server_key: "10b6c29efe77ea585c4242ffcf58bba6"
server_key_iv: "4ae2f1e31f404f83b5cf322cf7d87a85"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.264060 3334 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 22:15:39.267901 3357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.269367 3334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:15:39.269557 3334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "9069c916037c45fbb1be6e67ddcadbed"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-68xk"
server_key: "10b6c29efe77ea585c4242ffcf58bba6"
server_key_iv: "4ae2f1e31f404f83b5cf322cf7d87a85"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.269773 3334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:39.284505 3334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:39.288731 3334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:39.289019 3334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:39.307322 3334 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:42607
I20260811 22:15:39.307343 3429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:42607 every 8 connection(s)
I20260811 22:15:39.309019 3334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:15:39.313423 3431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:39.317430 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 3334
I20260811 22:15:39.317653 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:15:39.318343 10934 external_mini_cluster.cc:1428] Setting key 3a9ce8b4d45dc072766868d5e572918c
I20260811 22:15:39.324723 3431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Bootstrap starting.
I20260811 22:15:39.328552 3431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:39.329831 3431 log.cc:824] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:39.332731 3431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: No bootstrap required, opened a new log
Aug 11 22:15:39 dist-test-slave-68xk krb5kdc[3314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:39.341302 3431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:15:39.341711 3431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:39.341853 3431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9069c916037c45fbb1be6e67ddcadbed, State: Initialized, Role: FOLLOWER
I20260811 22:15:39.342645 3431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:15:39.342917 3431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:15:39.343094 3431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:15:39.343286 3431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:39.343370 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.320369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.10.173.190:42607)
0811 22:15:39.321184 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:39.321195 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:39.321242 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:15:39.323308 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:39.323334 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:39.323722 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:39.324043 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:39.326048 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.327490 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.329545 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.330378 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.330673 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.335407 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:39.335416 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:39.335431 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:39.335478 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:39.338877 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.339631 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.339640 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.339658 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.339839 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.340271 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.340276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.340280 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.340840 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:39.341145 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:39.341630 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 22:15:39.342110 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":398,"threads_started":1}
I20260811 22:15:39.346289 3431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:15:39.346930 3431 leader_election.cc:302] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9069c916037c45fbb1be6e67ddcadbed; no voters:
I20260811 22:15:39.347522 3431 leader_election.cc:288] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:15:39.347676 3438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:39.348358 3438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 1 LEADER]: Becoming Leader. State: Replica: 9069c916037c45fbb1be6e67ddcadbed, State: Running, Role: LEADER
I20260811 22:15:39.348865 3438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:15:39.349630 3431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:15:39.352205 3442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9069c916037c45fbb1be6e67ddcadbed. Latest consensus state: current_term: 1 leader_uuid: "9069c916037c45fbb1be6e67ddcadbed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } } }
I20260811 22:15:39.352674 3442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:39.353037 3440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9069c916037c45fbb1be6e67ddcadbed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } } }
I20260811 22:15:39.353386 3440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: This master's current role is: LEADER
I20260811 22:15:39.355029 3448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:15:39.360716 3448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:15:39.370787 3448 catalog_manager.cc:1347] Generated new cluster ID: 955e4bc07f5e4550a2b1a152a939831c
I20260811 22:15:39.370915 3448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:15:39.380707 3448 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:15:39.382112 3448 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:15:39.388033 3448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Generated new TSK 0
I20260811 22:15:39.388962 3448 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:15:39.463109 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:42607
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:39.818310 3465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:39.818740 3465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:39.818918 3465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:39.826792 3465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:39.826946 3465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:39.827136 3465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:15:39.837056 3465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:42607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:39.838438 3465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:39.839854 3465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:39.854184 3475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.854954 3480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:39.854975 3476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:39.855408 3465 server_base.cc:1034] running on GCE node
I20260811 22:15:39.856014 3465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:39.856976 3465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:39.858176 3465 hybrid_clock.cc:648] HybridClock initialized: now 1786486539858134 us; error 35 us; skew 500 ppm
I20260811 22:15:39.858592 3465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:39 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486539, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:39.862016 3465 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:15:39.863503 3465 webserver.cc:460] Webserver started at http://127.10.173.129:33441/ using document root <none> and password file <none>
I20260811 22:15:39.864223 3465 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:39.864392 3465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:39.864719 3465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:39.867695 3465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "4671291604d3438e90e5bbdc94701fae"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-68xk"
server_key: "e4b6e6049feae08743a5eea58606afcc"
server_key_iv: "6be02ce6fbd28113391002a10b0f344e"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.868471 3465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "4671291604d3438e90e5bbdc94701fae"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-68xk"
server_key: "e4b6e6049feae08743a5eea58606afcc"
server_key_iv: "6be02ce6fbd28113391002a10b0f344e"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.873880 3465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 22:15:39.877339 3495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.878669 3465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:15:39.878854 3465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "4671291604d3438e90e5bbdc94701fae"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-68xk"
server_key: "e4b6e6049feae08743a5eea58606afcc"
server_key_iv: "6be02ce6fbd28113391002a10b0f344e"
server_key_version: "encryptionkey@0"
I20260811 22:15:39.879052 3465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:39.905189 3465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:39.909632 3465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:39.909957 3465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:39.911062 3465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:39.912441 3465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:39.912549 3465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.912763 3465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:39.912848 3465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:39.947829 3465 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:44223
I20260811 22:15:39.947889 3658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:44223 every 8 connection(s)
I20260811 22:15:39.949236 3465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:15:39.956408 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 3465
I20260811 22:15:39.956692 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:15:39.957475 10934 external_mini_cluster.cc:1428] Setting key ce9ccc2eb5c0caad698fc48fac2c85e6
Aug 11 22:15:39 dist-test-slave-68xk krb5kdc[3314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486539, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:39.971253 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.952324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:46259 (local address 127.10.173.190:42607)
0811 22:15:39.952820 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:39.952825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:39.953904 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 22:15:39.955934 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:39.955939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:39.956059 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:39.956184 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:39.957923 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.959055 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.961077 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.961500 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.961641 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.965303 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:39.965309 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:39.965313 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:39.965344 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:39.968024 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.968764 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.968771 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.968774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.968902 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:39.969344 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:39.969350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:39.969354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:39.969622 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:39.969784 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:39.970629 (+ 845us) server_negotiation.cc:299] Negotiation successful
0811 22:15:39.970864 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":250,"threads_started":1}
I20260811 22:15:39.972468 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:39.952984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:46259)
0811 22:15:39.953730 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:39.953776 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:39.955647 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:39.956441 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:39.956450 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:39.957079 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:39.957740 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.957760 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.959257 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.959262 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.960923 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:39.960936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:39.961761 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:39.961769 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:39.962014 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:39.962814 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:39.962838 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:39.965067 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:39.968175 (+ 3108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:39.968185 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:39.968202 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:39.968629 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:39.969021 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:39.969026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:39.969030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:39.969225 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:39.969766 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:39.969774 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:39.970180 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 22:15:39.970525 (+ 345us) client_negotiation.cc:240] Negotiation successful
0811 22:15:39.971073 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":301,"threads_started":1}
I20260811 22:15:39.974385 3659 heartbeater.cc:347] Connected to a master server at 127.10.173.190:42607
I20260811 22:15:39.974741 3659 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:39.975450 3659 heartbeater.cc:511] Master 127.10.173.190:42607 requested a full tablet report, sending...
I20260811 22:15:39.977648 3379 ts_manager.cc:194] Registered new tserver with Master: 4671291604d3438e90e5bbdc94701fae (127.10.173.129:44223)
I20260811 22:15:39.979185 3379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:46259
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:15:40.030164 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:42607
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:40.382993 3670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:40.383253 3670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:40.383415 3670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:40.390647 3670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:40.390750 3670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:40.390956 3670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:15:40.400645 3670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:42607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:40.402009 3670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:40.403383 3670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:15:40.416863 3677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:40.416896 3678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:40.416944 3682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:40.417889 3670 server_base.cc:1034] running on GCE node
I20260811 22:15:40.418613 3670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:40.419574 3670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:40.420790 3670 hybrid_clock.cc:648] HybridClock initialized: now 1786486540420738 us; error 37 us; skew 500 ppm
I20260811 22:15:40.421118 3670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:40 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486540, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:40.424464 3670 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:15:40.425997 3670 webserver.cc:460] Webserver started at http://127.10.173.130:36445/ using document root <none> and password file <none>
I20260811 22:15:40.426749 3670 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:40.426906 3670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:40.427270 3670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:40.430274 3670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "f5bf427196554e70b62d858f83ce80dc"
format_stamp: "Formatted at 2026-08-11 22:15:40 on dist-test-slave-68xk"
server_key: "726fd98158e559610a0f3076441850d7"
server_key_iv: "f6bbe2e6669ed71962d86a5b3fc68527"
server_key_version: "encryptionkey@0"
I20260811 22:15:40.431160 3670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "f5bf427196554e70b62d858f83ce80dc"
format_stamp: "Formatted at 2026-08-11 22:15:40 on dist-test-slave-68xk"
server_key: "726fd98158e559610a0f3076441850d7"
server_key_iv: "f6bbe2e6669ed71962d86a5b3fc68527"
server_key_version: "encryptionkey@0"
I20260811 22:15:40.436528 3670 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:15:40.440331 3696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:40.441645 3670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:15:40.441826 3670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "f5bf427196554e70b62d858f83ce80dc"
format_stamp: "Formatted at 2026-08-11 22:15:40 on dist-test-slave-68xk"
server_key: "726fd98158e559610a0f3076441850d7"
server_key_iv: "f6bbe2e6669ed71962d86a5b3fc68527"
server_key_version: "encryptionkey@0"
I20260811 22:15:40.442036 3670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:40.471194 3670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:40.475888 3670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:40.476191 3670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:40.477159 3670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:40.478578 3670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:40.478682 3670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:40.478844 3670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:40.478920 3670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:40.514258 3670 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:34559
I20260811 22:15:40.514313 3877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:34559 every 8 connection(s)
I20260811 22:15:40.515617 3670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:15:40.521203 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 3670
I20260811 22:15:40.521406 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:15:40.521884 10934 external_mini_cluster.cc:1428] Setting key 5845f3ab72cf734b20251a5c6e327afd
Aug 11 22:15:40 dist-test-slave-68xk krb5kdc[3314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486540, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:40.536994 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:40.518472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:52981 (local address 127.10.173.190:42607)
0811 22:15:40.518923 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:40.518928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:40.520029 (+ 1101us) server_negotiation.cc:407] Connection header received
0811 22:15:40.522149 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:40.522154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:40.522259 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:40.522384 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:40.524324 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.525061 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.526697 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.527123 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.527261 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:40.531022 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:40.531028 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:40.531031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:40.531063 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:40.533775 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:40.534581 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:40.534585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:40.534587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:40.534731 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:40.535209 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:40.535213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:40.535215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:40.535492 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:40.535658 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:40.536371 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 22:15:40.536605 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":223,"threads_started":1}
I20260811 22:15:40.538024 3884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:40.519072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:52981)
0811 22:15:40.519841 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:40.519888 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:40.521825 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:40.522673 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:40.522681 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:40.523415 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:40.524147 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:40.524166 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.525230 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.525234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:40.526559 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:40.526571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:40.527369 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:40.527377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:40.527606 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:40.528423 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:40.528447 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:40.530773 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:40.533924 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:40.533934 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:40.533953 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:40.534397 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:40.534850 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:40.534855 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:40.534858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:40.535069 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:40.535630 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:40.535638 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:40.535990 (+ 352us) client_negotiation.cc:769] Sending connection context
0811 22:15:40.536263 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 22:15:40.536807 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":324,"threads_started":1}
I20260811 22:15:40.539808 3880 heartbeater.cc:347] Connected to a master server at 127.10.173.190:42607
I20260811 22:15:40.540136 3880 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:40.540887 3880 heartbeater.cc:511] Master 127.10.173.190:42607 requested a full tablet report, sending...
I20260811 22:15:40.542508 3379 ts_manager.cc:194] Registered new tserver with Master: f5bf427196554e70b62d858f83ce80dc (127.10.173.130:34559)
I20260811 22:15:40.543839 3379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:52981
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:15:40.591476 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:42607
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:15:40.958743 3887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:15:40.959012 3887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:15:40.959111 3887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:15:40.966779 3887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:15:40.966892 3887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:15:40.967005 3887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:15:40.976998 3887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:42607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:15:40.978251 3887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:15:40.979600 3887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:15:40.982120 3659 heartbeater.cc:503] Master 127.10.173.190:42607 was elected leader, sending a full tablet report...
W20260811 22:15:40.993111 3899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:40.993211 3893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:15:40.993191 3892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:15:40.999472 3887 server_base.cc:1034] running on GCE node
I20260811 22:15:41.000233 3887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:15:41.001281 3887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:15:41.002569 3887 hybrid_clock.cc:648] HybridClock initialized: now 1786486541002507 us; error 53 us; skew 500 ppm
I20260811 22:15:41.002921 3887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:15:41 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486541, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:15:41.010607 3887 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:15:41.012732 3887 webserver.cc:460] Webserver started at http://127.10.173.131:42121/ using document root <none> and password file <none>
I20260811 22:15:41.013607 3887 fs_manager.cc:359] Metadata directory not provided
I20260811 22:15:41.013792 3887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:15:41.014183 3887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:15:41.017263 3887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "35d0121d93d94817924e71f0d0de5ceb"
format_stamp: "Formatted at 2026-08-11 22:15:41 on dist-test-slave-68xk"
server_key: "d9d5ab67dbcaa11f7850bcb114f1c80d"
server_key_iv: "b8c31405824c64ae87d5fdfe17702afa"
server_key_version: "encryptionkey@0"
I20260811 22:15:41.018010 3887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "35d0121d93d94817924e71f0d0de5ceb"
format_stamp: "Formatted at 2026-08-11 22:15:41 on dist-test-slave-68xk"
server_key: "d9d5ab67dbcaa11f7850bcb114f1c80d"
server_key_iv: "b8c31405824c64ae87d5fdfe17702afa"
server_key_version: "encryptionkey@0"
I20260811 22:15:41.023496 3887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 22:15:41.027056 3912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:41.028517 3887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:15:41.028740 3887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "35d0121d93d94817924e71f0d0de5ceb"
format_stamp: "Formatted at 2026-08-11 22:15:41 on dist-test-slave-68xk"
server_key: "d9d5ab67dbcaa11f7850bcb114f1c80d"
server_key_iv: "b8c31405824c64ae87d5fdfe17702afa"
server_key_version: "encryptionkey@0"
I20260811 22:15:41.028976 3887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:15:41.072222 3887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:15:41.076807 3887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:15:41.077111 3887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:15:41.078156 3887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:15:41.079584 3887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:15:41.079708 3887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:41.079859 3887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:15:41.079932 3887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:15:41.115914 3887 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:43901
I20260811 22:15:41.116008 4070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:43901 every 8 connection(s)
I20260811 22:15:41.117297 3887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:15:41.122740 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 3887
I20260811 22:15:41.122959 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:15:41.123502 10934 external_mini_cluster.cc:1428] Setting key f3ff814df1e08b35527a969b3edbe227
Aug 11 22:15:41 dist-test-slave-68xk krb5kdc[3314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486541, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:15:41.141062 4075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.120465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:55043)
0811 22:15:41.121168 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.121214 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:41.123122 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:41.123879 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:41.123887 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:41.124415 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:15:41.125123 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.125140 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.126305 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.126310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.127584 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.127597 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.128380 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.128387 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.128611 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.129286 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:15:41.129308 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:15:41.131478 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:15:41.135490 (+ 4012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:41.135501 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:41.135520 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:41.136227 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:41.136719 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:15:41.136726 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:15:41.136729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:15:41.136931 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:15:41.137696 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:15:41.137705 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:15:41.138299 (+ 594us) client_negotiation.cc:769] Sending connection context
0811 22:15:41.138679 (+ 380us) client_negotiation.cc:240] Negotiation successful
0811 22:15:41.139317 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":284,"threads_started":1}
I20260811 22:15:41.142390 4074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.120065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:55043 (local address 127.10.173.190:42607)
0811 22:15:41.120562 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.120567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.121344 (+ 777us) server_negotiation.cc:407] Connection header received
0811 22:15:41.123392 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.123397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.123497 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.123624 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:41.125303 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.126132 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.127744 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.128143 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.128293 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.131692 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:41.131698 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:41.131701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:41.131735 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:41.135311 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:41.136394 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:41.136401 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:41.136406 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:41.136600 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:41.137060 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:41.137066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:41.137071 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:41.137515 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:41.137705 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.141672 (+ 3967us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.141918 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":255,"threads_started":1}
I20260811 22:15:41.147130 4071 heartbeater.cc:347] Connected to a master server at 127.10.173.190:42607
I20260811 22:15:41.147500 4071 heartbeater.cc:464] Registering TS with master...
I20260811 22:15:41.148514 4071 heartbeater.cc:511] Master 127.10.173.190:42607 requested a full tablet report, sending...
I20260811 22:15:41.150490 3379 ts_manager.cc:194] Registered new tserver with Master: 35d0121d93d94817924e71f0d0de5ceb (127.10.173.131:43901)
I20260811 22:15:41.151908 3379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:55043
I20260811 22:15:41.160485 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:15:41.183140 4074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.166161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56074 (local address 127.10.173.190:42607)
0811 22:15:41.166382 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.166395 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.166973 (+ 578us) server_negotiation.cc:407] Connection header received
0811 22:15:41.167198 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.167212 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.167279 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.167420 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:41.168969 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.169708 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.171212 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.171884 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.172037 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.175414 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:41.175422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:41.175428 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:41.175474 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:41.179541 (+ 4067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:41.180530 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:41.180536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:41.180539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:41.180735 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:41.181259 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:41.181264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:41.181268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:41.181856 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:41.182108 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.182529 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.182842 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:15:41.188190 3379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:15:41.199697 3379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:15:41.234740 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.213615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35164 (local address 127.10.173.131:43901)
0811 22:15:41.226600 (+ 12985us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.226606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.226634 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:15:41.226715 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.226720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.226907 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.227110 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:41.227827 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.229223 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.231665 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.233373 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.233577 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.233759 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.234118 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.234305 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12821,"thread_start_us":12645,"threads_started":1}
I20260811 22:15:41.234787 4090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.213254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:43901 (local address 127.0.0.1:35164)
0811 22:15:41.225094 (+ 11840us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.225146 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:41.225355 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:41.227121 (+ 1766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:41.227128 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:41.227161 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:41.227638 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.227649 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.229428 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.229433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.231501 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.231512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.233610 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.233616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.233812 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.233868 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 22:15:41.234012 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 22:15:41.234209 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11578,"spinlock_wait_cycles":27904,"thread_start_us":11422,"threads_started":1}
I20260811 22:15:41.237766 4091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.213297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:44852)
0811 22:15:41.225935 (+ 12638us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.225953 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:41.227222 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:41.228365 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:41.228372 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:41.228409 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:41.228949 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.228966 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.231250 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.231256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.233520 (+ 2264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.233539 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.236883 (+ 3344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.236890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.237080 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.237128 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 22:15:41.237217 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:15:41.237344 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12519,"mutex_wait_us":1,"thread_start_us":12365,"threads_started":1}
I20260811 22:15:41.237959 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.213406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44852 (local address 127.10.173.129:44223)
0811 22:15:41.224231 (+ 10825us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.224246 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.226065 (+ 1819us) server_negotiation.cc:407] Connection header received
0811 22:15:41.227497 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.227509 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.227811 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.228019 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:41.229208 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.231082 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.233829 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.236117 (+ 2288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.236369 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.236546 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.237433 (+ 887us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.237609 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10540,"thread_start_us":10841,"threads_started":1}
I20260811 22:15:41.239596 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.227156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:34559 (local address 127.0.0.1:37970)
0811 22:15:41.230973 (+ 3817us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.230992 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:41.231114 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:41.231761 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:41.231767 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:41.231796 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:41.232300 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.232313 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.234229 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.234237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.236277 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.236292 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.238762 (+ 2470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.238768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.238938 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.238966 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:15:41.239145 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 22:15:41.239274 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3681,"thread_start_us":3530,"threads_started":1}
I20260811 22:15:41.239622 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.226532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37970 (local address 127.10.173.130:34559)
0811 22:15:41.227096 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.227104 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.231123 (+ 4019us) server_negotiation.cc:407] Connection header received
0811 22:15:41.231284 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.231303 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.231555 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.231753 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:41.232515 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.234036 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.236463 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.238520 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.238718 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.238878 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.239104 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.239283 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":306,"threads_started":1}
I20260811 22:15:41.239085 3983 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0f59debbf442ac85a8e169af4a541f (DEFAULT_TABLE table=test-table [id=c15e25bb05934f8a8faaba08a61f0fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:41.240820 3983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0f59debbf442ac85a8e169af4a541f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:41.241803 3563 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0f59debbf442ac85a8e169af4a541f (DEFAULT_TABLE table=test-table [id=c15e25bb05934f8a8faaba08a61f0fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:41.243328 3563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0f59debbf442ac85a8e169af4a541f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:41.243105 3776 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0f59debbf442ac85a8e169af4a541f (DEFAULT_TABLE table=test-table [id=c15e25bb05934f8a8faaba08a61f0fb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:15:41.244210 3776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0f59debbf442ac85a8e169af4a541f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:15:41.256197 4103 tablet_bootstrap.cc:492] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: Bootstrap starting.
I20260811 22:15:41.257032 4104 tablet_bootstrap.cc:492] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: Bootstrap starting.
I20260811 22:15:41.258594 4105 tablet_bootstrap.cc:492] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Bootstrap starting.
I20260811 22:15:41.260663 4103 tablet_bootstrap.cc:654] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:41.261592 4105 tablet_bootstrap.cc:654] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:41.262020 4104 tablet_bootstrap.cc:654] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: Neither blocks nor log segments found. Creating new log.
I20260811 22:15:41.262019 4103 log.cc:824] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:41.262889 4105 log.cc:824] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:41.263374 4104 log.cc:824] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:15:41.265705 4103 tablet_bootstrap.cc:492] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: No bootstrap required, opened a new log
I20260811 22:15:41.265980 4103 ts_tablet_manager.cc:1397] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260811 22:15:41.266880 4104 tablet_bootstrap.cc:492] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: No bootstrap required, opened a new log
I20260811 22:15:41.267163 4104 ts_tablet_manager.cc:1397] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 22:15:41.267786 4105 tablet_bootstrap.cc:492] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: No bootstrap required, opened a new log
I20260811 22:15:41.268042 4105 ts_tablet_manager.cc:1397] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260811 22:15:41.274742 4104 raft_consensus.cc:348] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:15:41.275280 4104 raft_consensus.cc:374] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:41.275385 4104 raft_consensus.cc:729] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d0121d93d94817924e71f0d0de5ceb, State: Initialized, Role: FOLLOWER
I20260811 22:15:41.276003 4104 consensus_queue.cc:260] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:15:41.277112 4071 heartbeater.cc:503] Master 127.10.173.190:42607 was elected leader, sending a full tablet report...
I20260811 22:15:41.276899 4103 raft_consensus.cc:348] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:15:41.277335 4103 raft_consensus.cc:374] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:41.277422 4103 raft_consensus.cc:729] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5bf427196554e70b62d858f83ce80dc, State: Initialized, Role: FOLLOWER
I20260811 22:15:41.277817 4104 ts_tablet_manager.cc:1428] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 22:15:41.277982 4103 consensus_queue.cc:260] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:15:41.278656 4105 raft_consensus.cc:348] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:15:41.279122 4105 raft_consensus.cc:374] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:15:41.279218 4105 raft_consensus.cc:729] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4671291604d3438e90e5bbdc94701fae, State: Initialized, Role: FOLLOWER
I20260811 22:15:41.279417 3880 heartbeater.cc:503] Master 127.10.173.190:42607 was elected leader, sending a full tablet report...
I20260811 22:15:41.279868 4105 consensus_queue.cc:260] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:15:41.280133 4103 ts_tablet_manager.cc:1428] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260811 22:15:41.282853 4105 ts_tablet_manager.cc:1428] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
W20260811 22:15:41.369154 4073 tablet.cc:2369] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:41.413347 4112 raft_consensus.cc:482] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:15:41.413748 4112 raft_consensus.cc:504] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:15:41.416010 4112 leader_election.cc:288] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4671291604d3438e90e5bbdc94701fae (127.10.173.129:44223), f5bf427196554e70b62d858f83ce80dc (127.10.173.130:34559)
I20260811 22:15:41.422457 4075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.416317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.10.173.131:41399)
0811 22:15:41.416550 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.416595 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:41.417339 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:41.417854 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:41.417863 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:41.417891 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:41.418064 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.418073 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.419215 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.419219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.420566 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.420578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.421687 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.421693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.421880 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.421905 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:15:41.422014 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:15:41.422154 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 22:15:41.422466 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.416561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:41399 (local address 127.10.173.129:44223)
0811 22:15:41.416895 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.416904 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.416927 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:15:41.417565 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.417573 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.417682 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.417786 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:41.418239 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.419054 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.420727 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.421529 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.421700 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.421776 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.422041 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.422168 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 22:15:41.422868 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.417159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:41721 (local address 127.10.173.130:34559)
0811 22:15:41.417519 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.417525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.417674 (+ 149us) server_negotiation.cc:407] Connection header received
0811 22:15:41.417887 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.417899 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.417962 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.418011 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:15:41.418731 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.419438 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.421153 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.421918 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.422093 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.422165 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.422470 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.422624 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260811 22:15:41.423243 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.416670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:34559 (local address 127.10.173.131:41721)
0811 22:15:41.417524 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.417543 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:15:41.417704 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:15:41.418172 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:15:41.418175 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:15:41.418194 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:15:41.418577 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.418587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.419610 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.419615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.420977 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:15:41.420988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.422131 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.422137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:15:41.422289 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.422316 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:15:41.422423 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:15:41.422898 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":471,"threads_started":1}
I20260811 22:15:41.423664 3596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0f59debbf442ac85a8e169af4a541f" candidate_uuid: "35d0121d93d94817924e71f0d0de5ceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4671291604d3438e90e5bbdc94701fae" is_pre_election: true
I20260811 22:15:41.424242 3596 raft_consensus.cc:2454] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d0121d93d94817924e71f0d0de5ceb in term 0.
I20260811 22:15:41.424224 3804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0f59debbf442ac85a8e169af4a541f" candidate_uuid: "35d0121d93d94817924e71f0d0de5ceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5bf427196554e70b62d858f83ce80dc" is_pre_election: true
I20260811 22:15:41.424746 3804 raft_consensus.cc:2454] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35d0121d93d94817924e71f0d0de5ceb in term 0.
I20260811 22:15:41.425016 3917 leader_election.cc:302] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d0121d93d94817924e71f0d0de5ceb, 4671291604d3438e90e5bbdc94701fae; no voters:
I20260811 22:15:41.425432 4112 raft_consensus.cc:2781] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:15:41.425587 4112 raft_consensus.cc:482] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:15:41.425715 4112 raft_consensus.cc:3037] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:41.428037 4112 raft_consensus.cc:504] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:15:41.428663 4112 leader_election.cc:288] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [CANDIDATE]: Term 1 election: Requested vote from peers 4671291604d3438e90e5bbdc94701fae (127.10.173.129:44223), f5bf427196554e70b62d858f83ce80dc (127.10.173.130:34559)
I20260811 22:15:41.429136 3596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0f59debbf442ac85a8e169af4a541f" candidate_uuid: "35d0121d93d94817924e71f0d0de5ceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4671291604d3438e90e5bbdc94701fae"
I20260811 22:15:41.429206 3804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0f59debbf442ac85a8e169af4a541f" candidate_uuid: "35d0121d93d94817924e71f0d0de5ceb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5bf427196554e70b62d858f83ce80dc"
I20260811 22:15:41.429385 3596 raft_consensus.cc:3037] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:41.429423 3804 raft_consensus.cc:3037] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:15:41.431756 3804 raft_consensus.cc:2454] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35d0121d93d94817924e71f0d0de5ceb in term 1.
I20260811 22:15:41.431763 3596 raft_consensus.cc:2454] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35d0121d93d94817924e71f0d0de5ceb in term 1.
I20260811 22:15:41.432219 3917 leader_election.cc:302] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d0121d93d94817924e71f0d0de5ceb, f5bf427196554e70b62d858f83ce80dc; no voters:
I20260811 22:15:41.432548 4112 raft_consensus.cc:2781] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:15:41.433041 4112 raft_consensus.cc:686] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 1 LEADER]: Becoming Leader. State: Replica: 35d0121d93d94817924e71f0d0de5ceb, State: Running, Role: LEADER
I20260811 22:15:41.433941 4112 consensus_queue.cc:237] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:15:41.438588 3378 catalog_manager.cc:5495] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb reported cstate change: term changed from 0 to 1, leader changed from <none> to 35d0121d93d94817924e71f0d0de5ceb (127.10.173.131). New cstate: current_term: 1 leader_uuid: "35d0121d93d94817924e71f0d0de5ceb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } health_report { overall_health: UNKNOWN } } }
W20260811 22:15:41.451975 3662 tablet.cc:2369] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:15:41.473964 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.468491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33296 (local address 127.10.173.131:43901)
0811 22:15:41.468686 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.468691 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.468725 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:15:41.468910 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.468913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.469001 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.469130 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:15:41.469499 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.470220 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.471663 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.472079 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.472177 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.472646 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:15:41.473246 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:15:41.473506 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.473591 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.473659 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:15:41.481987 3804 raft_consensus.cc:1261] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc [term 1 FOLLOWER]: Refusing update from remote peer 35d0121d93d94817924e71f0d0de5ceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:41.482036 3596 raft_consensus.cc:1261] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae [term 1 FOLLOWER]: Refusing update from remote peer 35d0121d93d94817924e71f0d0de5ceb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:15:41.483032 4118 consensus_queue.cc:1035] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:41.483402 4112 consensus_queue.cc:1035] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:15:41.496224 4123 mvcc.cc:205] Tried to move back new op lower bound from 7317448873900675072 to 7317448873717641216. Current Snapshot: MvccSnapshot[applied={T|T < 7317448873900675072}]
I20260811 22:15:41.509654 3378 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:41.509948 3378 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:15:41.513595 3378 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Sending DeleteTablet for 3 replicas of tablet 0e0f59debbf442ac85a8e169af4a541f
I20260811 22:15:41.515008 3563 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0f59debbf442ac85a8e169af4a541f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:41 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:44852
I20260811 22:15:41.515069 3776 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0f59debbf442ac85a8e169af4a541f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:41 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:37970
I20260811 22:15:41.516232 3983 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0f59debbf442ac85a8e169af4a541f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:15:41 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:35164
I20260811 22:15:41.516364 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 3465
I20260811 22:15:41.540207 4148 tablet_replica.cc:330] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: stopping tablet replica
I20260811 22:15:41.540949 4148 raft_consensus.cc:2229] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:15:41.541859 4148 raft_consensus.cc:2258] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:41.544588 4148 ts_tablet_manager.cc:1905] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:41.545614 4144 tablet_replica.cc:330] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: stopping tablet replica
I20260811 22:15:41.546239 4138 tablet_replica.cc:330] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: stopping tablet replica
I20260811 22:15:41.546756 4144 raft_consensus.cc:2229] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:41.547353 4138 raft_consensus.cc:2229] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:15:41.550335 4144 raft_consensus.cc:2258] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:41.551043 4138 raft_consensus.cc:2258] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:15:41.553206 4144 ts_tablet_manager.cc:1905] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:15:41.553993 4138 ts_tablet_manager.cc:1905] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 22:15:41 dist-test-slave-68xk krb5kdc[3314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:15:41.563925 4148 ts_tablet_manager.cc:1918] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:41.566040 4092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.517676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.10.173.129:44223)
0811 22:15:41.517904 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.517912 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.541049 (+ 23137us) server_negotiation.cc:407] Connection header received
0811 22:15:41.541674 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.541682 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.541778 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.541886 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:41.545567 (+ 3681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.546942 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.549220 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.550857 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.551014 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.557262 (+ 6248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:41.557278 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:41.557296 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:41.557346 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:41.561811 (+ 4465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:41.562847 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:41.562856 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:41.562860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:41.563115 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:41.563660 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:41.563667 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:41.563670 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:41.564356 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:41.564645 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.565327 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.565571 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:15:41.566121 4144 ts_tablet_manager.cc:1918] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:41.566277 4144 log.cc:1196] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/0e0f59debbf442ac85a8e169af4a541f
I20260811 22:15:41.566548 4148 log.cc:1196] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/0e0f59debbf442ac85a8e169af4a541f
I20260811 22:15:41.567101 4148 ts_tablet_manager.cc:1939] T 0e0f59debbf442ac85a8e169af4a541f P 35d0121d93d94817924e71f0d0de5ceb: Deleting consensus metadata
I20260811 22:15:41.568002 3654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:41.569015 3362 catalog_manager.cc:4841] TS 35d0121d93d94817924e71f0d0de5ceb (127.10.173.131:43901): tablet 0e0f59debbf442ac85a8e169af4a541f (table test-table [id=c15e25bb05934f8a8faaba08a61f0fb9]) successfully deleted
I20260811 22:15:41.569309 4144 ts_tablet_manager.cc:1939] T 0e0f59debbf442ac85a8e169af4a541f P f5bf427196554e70b62d858f83ce80dc: Deleting consensus metadata
I20260811 22:15:41.571043 3363 catalog_manager.cc:4841] TS f5bf427196554e70b62d858f83ce80dc (127.10.173.130:34559): tablet 0e0f59debbf442ac85a8e169af4a541f (table test-table [id=c15e25bb05934f8a8faaba08a61f0fb9]) successfully deleted
I20260811 22:15:41.660131 4138 ts_tablet_manager.cc:1918] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:15:41.660465 4138 log.cc:1196] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/0e0f59debbf442ac85a8e169af4a541f
I20260811 22:15:41.660766 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 3465
W20260811 22:15:41.698040 3363 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.173.129:44223 (error 108)
I20260811 22:15:41.698498 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 3670
I20260811 22:15:41.700616 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.700052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:43698)
0811 22:15:41.700298 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.700425 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 22:15:41.700927 3363 catalog_manager.cc:4584] TS 4671291604d3438e90e5bbdc94701fae (127.10.173.129:44223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e0f59debbf442ac85a8e169af4a541f: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Aug 11 22:15:41 dist-test-slave-68xk krb5kdc[3314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:15:41.716853 4100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.699816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52928 (local address 127.10.173.130:34559)
0811 22:15:41.700004 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.700011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.700031 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:15:41.700101 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.700107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.700199 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.700311 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:41.701909 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.703114 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.704894 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.705533 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.705659 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.710088 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:41.710106 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:41.710126 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:41.710172 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:41.713928 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:41.714693 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:41.714702 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:41.714706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:41.714875 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:41.715213 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:41.715218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:41.715221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:41.715729 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:41.716041 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.716236 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.716468 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:15:41.717756 3870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:41.754729 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.754123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:43706)
0811 22:15:41.754385 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.754537 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 22:15:41.756008 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.755583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:43708)
0811 22:15:41.755758 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.755850 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:15:41.809903 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 3670
I20260811 22:15:41.816927 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.816331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:43716)
0811 22:15:41.816585 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.816706 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 22:15:41.818274 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.817831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:43724)
0811 22:15:41.818025 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.818123 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:15:41.833318 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 3887
Aug 11 22:15:41 dist-test-slave-68xk krb5kdc[3314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:15:41.857157 4097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.834476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33298 (local address 127.10.173.131:43901)
0811 22:15:41.834667 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:15:41.834671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:15:41.834691 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:15:41.834789 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:15:41.834793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:15:41.834877 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:15:41.834968 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:15:41.836853 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.837678 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.839718 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:15:41.840202 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:15:41.840332 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:15:41.850213 (+ 9881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:15:41.850228 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:15:41.850245 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:15:41.850303 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:15:41.853829 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:41.854862 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:41.854870 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:41.854872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:41.855062 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:15:41.855528 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:15:41.855532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:15:41.855534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:15:41.855898 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:15:41.856197 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 22:15:41.856621 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 22:15:41.856816 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:15:41.857905 4066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:41.899466 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.898865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:43736)
0811 22:15:41.899116 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.899236 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 22:15:41.900672 4102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:15:41.900180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:43738)
0811 22:15:41.900440 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 22:15:41.900526 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260811 22:15:41.986234 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 3887
I20260811 22:15:42.017191 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 3334
I20260811 22:15:42.018484 3424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:15:42.129307 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 3334
I20260811 22:16:02.157635 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:42607
--webserver_interface=127.10.173.190
--webserver_port=36385
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:42607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:02.516275 4158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:02.516569 4158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:02.516662 4158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:02.523797 4158 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:16:02.523902 4158 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:16:02.523943 4158 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:16:02.523989 4158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:16:02.524046 4158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:02.524109 4158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:02.524155 4158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:16:02.524204 4158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:16:02.534250 4158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:42607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:42607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=36385
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:02.535652 4158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:02.537089 4158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:02.563669 4174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:02.563687 4168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:02.563596 4169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:02.564575 4158 server_base.cc:1034] running on GCE node
I20260811 22:16:02.565593 4158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:02.567368 4158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:02.568642 4158 hybrid_clock.cc:648] HybridClock initialized: now 1786486562568537 us; error 71 us; skew 500 ppm
I20260811 22:16:02.569144 4158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:02 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486562, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:02.572924 4158 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:16:02.574726 4158 webserver.cc:460] Webserver started at http://127.10.173.190:36385/ using document root <none> and password file <none>
I20260811 22:16:02.575515 4158 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:02.575629 4158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:02.581955 4158 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:16:02.585263 4184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:02.586896 4158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:16:02.587092 4158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "9069c916037c45fbb1be6e67ddcadbed"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-68xk"
server_key: "10b6c29efe77ea585c4242ffcf58bba6"
server_key_iv: "4ae2f1e31f404f83b5cf322cf7d87a85"
server_key_version: "encryptionkey@0"
I20260811 22:16:02.588183 4158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:02.621214 4158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:02.626138 4158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:02.626509 4158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:02.647830 4158 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:42607
I20260811 22:16:02.647863 4259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:42607 every 8 connection(s)
I20260811 22:16:02.649434 4158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:16:02.651103 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 4158
I20260811 22:16:02.652267 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:44223
--local_ip_for_outbound_sockets=127.10.173.129
--tserver_master_addrs=127.10.173.190:42607
--webserver_port=33441
--webserver_interface=127.10.173.129
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:16:02.660370 4261 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:16:02.665743 4261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Bootstrap starting.
I20260811 22:16:02.688272 4261 log.cc:824] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:02.701987 4261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:16:02.702733 4261 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Bootstrap complete.
I20260811 22:16:02.713910 4261 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:16:02.714936 4261 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9069c916037c45fbb1be6e67ddcadbed, State: Initialized, Role: FOLLOWER
I20260811 22:16:02.715623 4261 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:16:02.715893 4261 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:16:02.716056 4261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:16:02.716270 4261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:16:02.720566 4261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:16:02.721148 4261 leader_election.cc:302] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9069c916037c45fbb1be6e67ddcadbed; no voters:
I20260811 22:16:02.721899 4261 leader_election.cc:288] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:16:02.722059 4266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:16:02.723433 4266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 2 LEADER]: Becoming Leader. State: Replica: 9069c916037c45fbb1be6e67ddcadbed, State: Running, Role: LEADER
I20260811 22:16:02.724117 4266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:16:02.724529 4261 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:16:02.727121 4268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9069c916037c45fbb1be6e67ddcadbed. Latest consensus state: current_term: 2 leader_uuid: "9069c916037c45fbb1be6e67ddcadbed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } } }
I20260811 22:16:02.727607 4268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:02.728343 4267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9069c916037c45fbb1be6e67ddcadbed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } } }
I20260811 22:16:02.728746 4267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:02.736510 4276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:16:02.745677 4276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:16:02.747495 4276 catalog_manager.cc:1259] Loaded cluster ID: 955e4bc07f5e4550a2b1a152a939831c
I20260811 22:16:02.747665 4276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:16:02.752138 4276 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:16:02.753861 4276 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Loaded TSK: 0
I20260811 22:16:02.798444 4276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Generated new TSK 1
I20260811 22:16:02.798959 4276 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:16:03.039870 4264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:03.040148 4264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:03.040306 4264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:03.047686 4264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:03.047793 4264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:03.047959 4264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:16:03.057848 4264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:44223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=33441
--tserver_master_addrs=127.10.173.190:42607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:03.059249 4264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:03.060624 4264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:03.076766 4302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:03.076978 4303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:03.077605 4305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:03.078540 4264 server_base.cc:1034] running on GCE node
I20260811 22:16:03.079396 4264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:03.080772 4264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:03.082072 4264 hybrid_clock.cc:648] HybridClock initialized: now 1786486563082012 us; error 68 us; skew 500 ppm
I20260811 22:16:03.082545 4264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:03 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486563, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:03.087147 4264 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:16:03.088917 4264 webserver.cc:460] Webserver started at http://127.10.173.129:33441/ using document root <none> and password file <none>
I20260811 22:16:03.089721 4264 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:03.089907 4264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:03.096443 4264 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:16:03.099779 4318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:03.101300 4264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:16:03.101516 4264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "4671291604d3438e90e5bbdc94701fae"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-68xk"
server_key: "e4b6e6049feae08743a5eea58606afcc"
server_key_iv: "6be02ce6fbd28113391002a10b0f344e"
server_key_version: "encryptionkey@0"
I20260811 22:16:03.102620 4264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:03.125414 4264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:03.130280 4264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:03.130713 4264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:03.131829 4264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:03.133888 4328 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 22:16:03.141850 4328 ts_tablet_manager.cc:1852] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 22:16:03.143379 4328 ts_tablet_manager.cc:1905] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:03.154063 4328 ts_tablet_manager.cc:1918] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:03.154335 4328 log.cc:1196] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/0e0f59debbf442ac85a8e169af4a541f
I20260811 22:16:03.154768 4328 ts_tablet_manager.cc:1939] T 0e0f59debbf442ac85a8e169af4a541f P 4671291604d3438e90e5bbdc94701fae: Deleting consensus metadata
I20260811 22:16:03.155226 4264 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 22:16:03.155385 4264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.022s user 0.001s sys 0.000s
I20260811 22:16:03.155584 4264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:03.155699 4264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:03.194917 4264 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:44223
I20260811 22:16:03.194967 4504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:44223 every 8 connection(s)
I20260811 22:16:03.196427 4264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:16:03.206964 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 4264
I20260811 22:16:03.207767 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:34559
--local_ip_for_outbound_sockets=127.10.173.130
--tserver_master_addrs=127.10.173.190:42607
--webserver_port=36445
--webserver_interface=127.10.173.130
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:16:03 dist-test-slave-68xk krb5kdc[3314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486563, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:03.222970 4507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:03.199383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:45285 (local address 127.10.173.190:42607)
0811 22:16:03.200089 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:03.200112 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:03.200650 (+ 538us) server_negotiation.cc:407] Connection header received
0811 22:16:03.202837 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:03.202851 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:03.203217 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:03.203536 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:03.204980 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:03.206602 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:03.208650 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:03.209867 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:03.210089 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:03.213522 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:03.213536 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:03.213571 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:03.213628 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:03.218168 (+ 4540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:03.218954 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:03.218966 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:03.218978 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:03.219215 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:03.219631 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:03.219639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:03.219646 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:03.220384 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:03.220582 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:03.221531 (+ 949us) server_negotiation.cc:299] Negotiation successful
0811 22:16:03.221890 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":269,"threads_started":1}
I20260811 22:16:03.223264 4508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:03.199613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:45285)
0811 22:16:03.200460 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:03.200497 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:03.202229 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:03.203659 (+ 1430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:03.203666 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:03.204186 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:03.204747 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:03.204765 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:03.206783 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:03.206787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:03.208497 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:03.208510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:03.210081 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:03.210086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:03.210276 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:03.211038 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:16:03.211061 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:03.213222 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:03.218323 (+ 5101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:03.218330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:03.218341 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:03.218813 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:03.219345 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:03.219349 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:03.219351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:03.219517 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:03.220567 (+ 1050us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:03.220576 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:03.221016 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 22:16:03.221345 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 22:16:03.221951 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":377,"threads_started":1}
I20260811 22:16:03.226459 4505 heartbeater.cc:347] Connected to a master server at 127.10.173.190:42607
I20260811 22:16:03.226789 4505 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:03.227697 4505 heartbeater.cc:511] Master 127.10.173.190:42607 requested a full tablet report, sending...
I20260811 22:16:03.230120 4207 ts_manager.cc:194] Registered new tserver with Master: 4671291604d3438e90e5bbdc94701fae (127.10.173.129:44223)
I20260811 22:16:03.231912 4207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:45285
W20260811 22:16:03.560700 4511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:03.561125 4511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:03.561340 4511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:03.569459 4511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:03.569563 4511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:03.569677 4511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:16:03.580384 4511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:34559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=36445
--tserver_master_addrs=127.10.173.190:42607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:03.582247 4511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:03.583663 4511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:03.600790 4519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:03.600867 4523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:03.601087 4511 server_base.cc:1034] running on GCE node
W20260811 22:16:03.600764 4518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:03.602088 4511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:03.603186 4511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:03.604441 4511 hybrid_clock.cc:648] HybridClock initialized: now 1786486563604389 us; error 36 us; skew 500 ppm
I20260811 22:16:03.604784 4511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:03 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486563, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:03.608217 4511 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:16:03.609769 4511 webserver.cc:460] Webserver started at http://127.10.173.130:36445/ using document root <none> and password file <none>
I20260811 22:16:03.610492 4511 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:03.610617 4511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:03.616118 4511 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 22:16:03.618992 4535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:03.620359 4511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:16:03.620502 4511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "f5bf427196554e70b62d858f83ce80dc"
format_stamp: "Formatted at 2026-08-11 22:15:40 on dist-test-slave-68xk"
server_key: "726fd98158e559610a0f3076441850d7"
server_key_iv: "f6bbe2e6669ed71962d86a5b3fc68527"
server_key_version: "encryptionkey@0"
I20260811 22:16:03.621464 4511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:03.654981 4511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:03.659803 4511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:03.660157 4511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:03.661284 4511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:03.662781 4511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:03.662921 4511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:03.663056 4511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:03.663153 4511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:03.699736 4511 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:34559
I20260811 22:16:03.699838 4711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:34559 every 8 connection(s)
I20260811 22:16:03.701262 4511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:16:03.704556 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 4511
I20260811 22:16:03.705246 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:43901
--local_ip_for_outbound_sockets=127.10.173.131
--tserver_master_addrs=127.10.173.190:42607
--webserver_port=42121
--webserver_interface=127.10.173.131
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:16:03 dist-test-slave-68xk krb5kdc[3314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486563, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:03.723904 4507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:03.704056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:56933 (local address 127.10.173.190:42607)
0811 22:16:03.704300 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:03.704308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:03.705800 (+ 1492us) server_negotiation.cc:407] Connection header received
0811 22:16:03.708113 (+ 2313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:03.708122 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:03.708255 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:03.708384 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:03.709962 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:03.711140 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:03.712854 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:03.713589 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:03.713737 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:03.717622 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:03.717628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:03.717631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:03.717669 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:03.720549 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:03.721335 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:03.721343 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:03.721347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:03.721486 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:03.721979 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:03.721986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:03.721990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:03.722300 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:03.722501 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:03.723252 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 22:16:03.723522 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:16:03.725415 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:03.704654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:56933)
0811 22:16:03.705608 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:03.705658 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:03.707825 (+ 2167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:03.708617 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:03.708625 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:03.709176 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:03.709775 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:03.709793 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:03.711347 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:03.711352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:03.712698 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:03.712711 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:03.713842 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:03.713851 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:03.714079 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:03.714829 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:16:03.714855 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:03.717355 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:03.720707 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:03.720716 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:03.720734 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:03.721199 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:03.721625 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:03.721632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:03.721637 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:03.721848 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:03.722491 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:03.722500 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:03.722853 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 22:16:03.723164 (+ 311us) client_negotiation.cc:240] Negotiation successful
0811 22:16:03.723724 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":407,"threads_started":1}
I20260811 22:16:03.727170 4714 heartbeater.cc:347] Connected to a master server at 127.10.173.190:42607
I20260811 22:16:03.727546 4714 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:03.728396 4714 heartbeater.cc:511] Master 127.10.173.190:42607 requested a full tablet report, sending...
I20260811 22:16:03.730120 4207 ts_manager.cc:194] Registered new tserver with Master: f5bf427196554e70b62d858f83ce80dc (127.10.173.130:34559)
I20260811 22:16:03.731550 4207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:56933
W20260811 22:16:04.069033 4717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:04.069335 4717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:04.069440 4717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:04.077566 4717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:04.077677 4717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:04.077790 4717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:16:04.087631 4717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:43901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=42121
--tserver_master_addrs=127.10.173.190:42607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:04.088900 4717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:04.090210 4717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:04.103729 4730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:04.103816 4725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:04.103848 4723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:04.105652 4717 server_base.cc:1034] running on GCE node
I20260811 22:16:04.106220 4717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:04.107272 4717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:04.108479 4717 hybrid_clock.cc:648] HybridClock initialized: now 1786486564108436 us; error 36 us; skew 500 ppm
I20260811 22:16:04.108763 4717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:04 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486564, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:04.112102 4717 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:16:04.113523 4717 webserver.cc:460] Webserver started at http://127.10.173.131:42121/ using document root <none> and password file <none>
I20260811 22:16:04.114213 4717 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:04.114362 4717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:04.119827 4717 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:16:04.122627 4743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:04.124111 4717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:16:04.124341 4717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "35d0121d93d94817924e71f0d0de5ceb"
format_stamp: "Formatted at 2026-08-11 22:15:41 on dist-test-slave-68xk"
server_key: "d9d5ab67dbcaa11f7850bcb114f1c80d"
server_key_iv: "b8c31405824c64ae87d5fdfe17702afa"
server_key_version: "encryptionkey@0"
I20260811 22:16:04.125435 4717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:04.156651 4717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:04.161314 4717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:04.161568 4717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:04.162712 4717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:04.164252 4717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:04.164372 4717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:04.164511 4717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:04.164608 4717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:04.199214 4717 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:43901
I20260811 22:16:04.199329 4910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:43901 every 8 connection(s)
I20260811 22:16:04.200649 4717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:16:04.208815 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 4717
Aug 11 22:16:04 dist-test-slave-68xk krb5kdc[3314](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486564, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:04.223726 4507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.203353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:43019 (local address 127.10.173.190:42607)
0811 22:16:04.203572 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.203580 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.204653 (+ 1073us) server_negotiation.cc:407] Connection header received
0811 22:16:04.206498 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.206504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.206593 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.206713 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:04.208213 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.209335 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.212841 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.213298 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.213469 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.217127 (+ 3658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:04.217136 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:04.217142 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:04.217184 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:04.220667 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:04.221385 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:04.221392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:04.221395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:04.221524 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:04.221985 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:04.221990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:04.221994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:04.222282 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:04.222490 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.223125 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.223340 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:16:04.224817 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.203735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:43019)
0811 22:16:04.204464 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:04.204507 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:04.206213 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:04.206951 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:04.206958 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:04.207453 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:04.208035 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.208059 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.210468 (+ 2409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.210474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.212693 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.212709 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.213526 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.213536 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.213805 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.214523 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:16:04.214547 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:04.216906 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:04.220821 (+ 3915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:04.220830 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:04.220846 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:04.221264 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:04.221665 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:04.221671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:04.221675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:04.221869 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:04.222439 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:04.222449 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:04.222782 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 22:16:04.223044 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 22:16:04.223569 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":321,"threads_started":1}
I20260811 22:16:04.225641 4919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.209876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42226 (local address 127.10.173.190:42607)
0811 22:16:04.210353 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.210357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.210586 (+ 229us) server_negotiation.cc:407] Connection header received
0811 22:16:04.210769 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.210772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.210828 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.210911 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:04.213079 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.213840 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.215328 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.215913 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.216053 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.219624 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:04.219633 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:04.219636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:04.219676 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:04.222457 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:04.223179 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:04.223184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:04.223189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:04.223332 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:04.223764 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:04.223768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:04.223770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:04.224132 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:04.224501 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.224891 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.225128 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":268,"threads_started":1}
I20260811 22:16:04.226739 4911 heartbeater.cc:347] Connected to a master server at 127.10.173.190:42607
I20260811 22:16:04.227051 4911 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:04.227855 4911 heartbeater.cc:511] Master 127.10.173.190:42607 requested a full tablet report, sending...
I20260811 22:16:04.229403 4207 ts_manager.cc:194] Registered new tserver with Master: 35d0121d93d94817924e71f0d0de5ceb (127.10.173.131:43901)
I20260811 22:16:04.230276 4207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:43019
I20260811 22:16:04.236142 4505 heartbeater.cc:503] Master 127.10.173.190:42607 was elected leader, sending a full tablet report...
I20260811 22:16:04.241017 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:16:04.247741 4919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.241981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.10.173.190:42607)
0811 22:16:04.242205 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.242209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.242657 (+ 448us) server_negotiation.cc:407] Connection header received
0811 22:16:04.242840 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.242844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.242976 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.243114 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:16:04.243580 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.244316 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.245853 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.246279 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.246377 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.246940 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:16:04.247212 (+ 272us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 22:16:04.247499 (+ 287us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42238: token expired
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 22:16:04.247887 4919 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42238: token expired
W20260811 22:16:04.248452 4082 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.173.190:42607)
I20260811 22:16:04.261570 4919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.248995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42252 (local address 127.10.173.190:42607)
0811 22:16:04.249153 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.249157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.249216 (+ 59us) server_negotiation.cc:407] Connection header received
0811 22:16:04.249322 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.249326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.249380 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.249454 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:04.250894 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.251575 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.252988 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.253398 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.253496 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.256175 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:04.256180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:04.256183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:04.256215 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:04.259009 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:04.259691 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:04.259695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:04.259697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:04.259826 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:04.260215 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:04.260220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:04.260223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:04.260549 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:04.260756 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.261145 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.261335 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:16:04.265415 4207 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:16:04.273696 4207 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:16:04.297888 4932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.286951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:46632)
0811 22:16:04.287999 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:04.288062 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:04.288282 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:04.289458 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:04.289465 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:04.289508 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:04.290028 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.290043 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.292185 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.292191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.294768 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.294788 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.296816 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.296824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.297040 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.297112 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 22:16:04.297233 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:16:04.297432 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":459,"threads_started":1}
I20260811 22:16:04.299623 4934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.287175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46632 (local address 127.10.173.129:44223)
0811 22:16:04.288939 (+ 1764us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.288947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.288971 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:16:04.289052 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.289058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.289240 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.289450 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:04.290394 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.292007 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.295006 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.296541 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.298833 (+ 2292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.299033 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.299109 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.299281 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1590,"thread_start_us":1478,"threads_started":1}
I20260811 22:16:04.302150 4399 tablet_service.cc:1467] Processing CreateTablet for tablet b2d434b1dd904b959efe0d73d15f4382 (DEFAULT_TABLE table=test-table [id=24799d4a947c4ace96fe5f8295a6f255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:04.303211 4933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.287114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:43901 (local address 127.0.0.1:56250)
0811 22:16:04.290105 (+ 2991us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:04.290125 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:04.290235 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:04.292536 (+ 2301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:04.292541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:04.292578 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:04.292976 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.292989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.296402 (+ 3413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.296408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.299828 (+ 3420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.299843 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.302305 (+ 2462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.302311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.302529 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.302564 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:16:04.302661 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 22:16:04.302798 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2857,"thread_start_us":2692,"threads_started":1}
I20260811 22:16:04.303272 4938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.288990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.10.173.131:43901)
0811 22:16:04.291573 (+ 2583us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.291582 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.291614 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:16:04.291716 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.291724 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.292004 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.292218 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:04.294543 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.296214 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.300035 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.301946 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.302138 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.302457 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.302731 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.302912 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2408,"thread_start_us":2257,"threads_started":1}
I20260811 22:16:04.303813 4399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d434b1dd904b959efe0d73d15f4382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:04.307269 4932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.288432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:34559 (local address 127.0.0.1:47196)
0811 22:16:04.298443 (+ 10011us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:04.298466 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:04.298569 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:04.300873 (+ 2304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:04.300878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:04.300908 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:04.301136 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.301146 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.303006 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.303012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.304555 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.304570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.306479 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.306485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.306669 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.306712 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:16:04.306785 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:16:04.306924 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9865,"thread_start_us":10769,"threads_started":1}
I20260811 22:16:04.307533 4824 tablet_service.cc:1467] Processing CreateTablet for tablet b2d434b1dd904b959efe0d73d15f4382 (DEFAULT_TABLE table=test-table [id=24799d4a947c4ace96fe5f8295a6f255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:04.308358 4941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.289814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.10.173.130:34559)
0811 22:16:04.300331 (+ 10517us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.300339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.300372 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:16:04.300465 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.300471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.300679 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.300896 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:04.301290 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.302852 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.304793 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.306252 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.307565 (+ 1313us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.307722 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.307798 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.307950 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10223,"thread_start_us":10087,"threads_started":1}
I20260811 22:16:04.308954 4824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d434b1dd904b959efe0d73d15f4382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:04.312227 4613 tablet_service.cc:1467] Processing CreateTablet for tablet b2d434b1dd904b959efe0d73d15f4382 (DEFAULT_TABLE table=test-table [id=24799d4a947c4ace96fe5f8295a6f255]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:04.313598 4613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2d434b1dd904b959efe0d73d15f4382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:04.314358 4944 tablet_bootstrap.cc:492] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Bootstrap starting.
I20260811 22:16:04.317504 4944 tablet_bootstrap.cc:654] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:04.318650 4944 log.cc:824] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:04.323798 4944 tablet_bootstrap.cc:492] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: No bootstrap required, opened a new log
I20260811 22:16:04.324090 4944 ts_tablet_manager.cc:1397] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 22:16:04.325315 4946 tablet_bootstrap.cc:492] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: Bootstrap starting.
I20260811 22:16:04.329082 4947 tablet_bootstrap.cc:492] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: Bootstrap starting.
I20260811 22:16:04.330919 4946 tablet_bootstrap.cc:654] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:04.332285 4946 log.cc:824] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:04.332334 4947 tablet_bootstrap.cc:654] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:04.333690 4947 log.cc:824] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:04.334966 4944 raft_consensus.cc:348] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:16:04.335441 4944 raft_consensus.cc:374] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:04.335554 4944 raft_consensus.cc:729] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4671291604d3438e90e5bbdc94701fae, State: Initialized, Role: FOLLOWER
I20260811 22:16:04.335891 4946 tablet_bootstrap.cc:492] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: No bootstrap required, opened a new log
I20260811 22:16:04.336159 4946 ts_tablet_manager.cc:1397] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260811 22:16:04.336223 4944 consensus_queue.cc:260] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:16:04.336582 4947 tablet_bootstrap.cc:492] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: No bootstrap required, opened a new log
I20260811 22:16:04.336850 4947 ts_tablet_manager.cc:1397] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 22:16:04.343873 4944 ts_tablet_manager.cc:1428] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Time spent starting tablet: real 0.020s user 0.010s sys 0.010s
I20260811 22:16:04.344915 4947 raft_consensus.cc:348] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:16:04.345430 4947 raft_consensus.cc:374] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:04.345541 4947 raft_consensus.cc:729] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5bf427196554e70b62d858f83ce80dc, State: Initialized, Role: FOLLOWER
I20260811 22:16:04.346292 4947 consensus_queue.cc:260] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:16:04.348047 4714 heartbeater.cc:503] Master 127.10.173.190:42607 was elected leader, sending a full tablet report...
I20260811 22:16:04.348670 4947 ts_tablet_manager.cc:1428] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260811 22:16:04.350124 4946 raft_consensus.cc:348] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:16:04.350539 4946 raft_consensus.cc:374] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:04.350670 4946 raft_consensus.cc:729] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d0121d93d94817924e71f0d0de5ceb, State: Initialized, Role: FOLLOWER
I20260811 22:16:04.351214 4946 consensus_queue.cc:260] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:16:04.352330 4911 heartbeater.cc:503] Master 127.10.173.190:42607 was elected leader, sending a full tablet report...
I20260811 22:16:04.353055 4946 ts_tablet_manager.cc:1428] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
W20260811 22:16:04.449069 4506 tablet.cc:2369] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:16:04.452282 4912 tablet.cc:2369] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:16:04.453404 4715 tablet.cc:2369] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:16:04.644131 4952 raft_consensus.cc:482] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:16:04.644510 4952 raft_consensus.cc:504] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:16:04.646642 4952 leader_election.cc:288] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4671291604d3438e90e5bbdc94701fae (127.10.173.129:44223), 35d0121d93d94817924e71f0d0de5ceb (127.10.173.131:43901)
I20260811 22:16:04.656085 4938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.647275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:56225 (local address 127.10.173.131:43901)
0811 22:16:04.647553 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.647565 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.648914 (+ 1349us) server_negotiation.cc:407] Connection header received
0811 22:16:04.649180 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.649186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.649274 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.649383 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:04.650079 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.651298 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.653568 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.654941 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.655137 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.655237 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.655622 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.655803 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:16:04.656108 4954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.647129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:43901 (local address 127.10.173.130:56225)
0811 22:16:04.648806 (+ 1677us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:04.648830 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:04.649016 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:04.649420 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:04.649425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:04.649451 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:04.649892 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.649903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.651462 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.651468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.653384 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.653399 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.655186 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.655192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.655396 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.655438 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 22:16:04.655573 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 22:16:04.655710 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1562,"mutex_wait_us":73,"thread_start_us":906,"threads_started":1}
I20260811 22:16:04.656284 4934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.647045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:50107 (local address 127.10.173.129:44223)
0811 22:16:04.647829 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.647838 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.648619 (+ 781us) server_negotiation.cc:407] Connection header received
0811 22:16:04.648960 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.648966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.649044 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.649138 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:04.649862 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.651182 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.653824 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.655211 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.655386 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.655474 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.655855 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.656002 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555}
I20260811 22:16:04.657054 4955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.646767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.10.173.130:50107)
0811 22:16:04.648474 (+ 1707us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:04.648501 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:04.648771 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:04.649198 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:04.649203 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:04.649228 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:04.649681 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.649695 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.651346 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.651352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.653649 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:04.653663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.655440 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.655448 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:04.655649 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.655686 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:16:04.656477 (+ 791us) client_negotiation.cc:240] Negotiation successful
0811 22:16:04.656633 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1513,"thread_start_us":1716,"threads_started":1}
I20260811 22:16:04.657649 4853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d434b1dd904b959efe0d73d15f4382" candidate_uuid: "f5bf427196554e70b62d858f83ce80dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35d0121d93d94817924e71f0d0de5ceb" is_pre_election: true
I20260811 22:16:04.658252 4853 raft_consensus.cc:2454] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5bf427196554e70b62d858f83ce80dc in term 0.
I20260811 22:16:04.658731 4431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d434b1dd904b959efe0d73d15f4382" candidate_uuid: "f5bf427196554e70b62d858f83ce80dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4671291604d3438e90e5bbdc94701fae" is_pre_election: true
I20260811 22:16:04.658987 4542 leader_election.cc:302] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35d0121d93d94817924e71f0d0de5ceb, f5bf427196554e70b62d858f83ce80dc; no voters:
I20260811 22:16:04.659363 4431 raft_consensus.cc:2454] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5bf427196554e70b62d858f83ce80dc in term 0.
I20260811 22:16:04.659471 4952 raft_consensus.cc:2781] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:16:04.659607 4952 raft_consensus.cc:482] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:16:04.659703 4952 raft_consensus.cc:3037] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:04.663007 4952 raft_consensus.cc:504] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:16:04.663743 4952 leader_election.cc:288] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [CANDIDATE]: Term 1 election: Requested vote from peers 4671291604d3438e90e5bbdc94701fae (127.10.173.129:44223), 35d0121d93d94817924e71f0d0de5ceb (127.10.173.131:43901)
I20260811 22:16:04.664284 4431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d434b1dd904b959efe0d73d15f4382" candidate_uuid: "f5bf427196554e70b62d858f83ce80dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4671291604d3438e90e5bbdc94701fae"
I20260811 22:16:04.664278 4853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2d434b1dd904b959efe0d73d15f4382" candidate_uuid: "f5bf427196554e70b62d858f83ce80dc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35d0121d93d94817924e71f0d0de5ceb"
I20260811 22:16:04.664492 4853 raft_consensus.cc:3037] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:04.664515 4431 raft_consensus.cc:3037] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:04.666829 4431 raft_consensus.cc:2454] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5bf427196554e70b62d858f83ce80dc in term 1.
I20260811 22:16:04.667280 4545 leader_election.cc:302] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4671291604d3438e90e5bbdc94701fae, f5bf427196554e70b62d858f83ce80dc; no voters:
I20260811 22:16:04.667635 4952 raft_consensus.cc:2781] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:16:04.667693 4853 raft_consensus.cc:2454] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5bf427196554e70b62d858f83ce80dc in term 1.
I20260811 22:16:04.668728 4952 raft_consensus.cc:686] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 1 LEADER]: Becoming Leader. State: Replica: f5bf427196554e70b62d858f83ce80dc, State: Running, Role: LEADER
I20260811 22:16:04.669237 4952 consensus_queue.cc:237] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } }
I20260811 22:16:04.673708 4207 catalog_manager.cc:5495] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc reported cstate change: term changed from 0 to 1, leader changed from <none> to f5bf427196554e70b62d858f83ce80dc (127.10.173.130). New cstate: current_term: 1 leader_uuid: "f5bf427196554e70b62d858f83ce80dc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5bf427196554e70b62d858f83ce80dc" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34559 } health_report { overall_health: HEALTHY } } }
I20260811 22:16:04.759490 4941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.754369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47204 (local address 127.10.173.130:34559)
0811 22:16:04.754583 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.754588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.754611 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:16:04.754705 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.754709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.754786 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.754917 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:16:04.755324 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.756094 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.757419 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.757863 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.757979 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.758460 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:16:04.759010 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:16:04.759163 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.759205 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.759276 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:16:04.768476 4431 raft_consensus.cc:1261] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae [term 1 FOLLOWER]: Refusing update from remote peer f5bf427196554e70b62d858f83ce80dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:16:04.768476 4853 raft_consensus.cc:1261] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb [term 1 FOLLOWER]: Refusing update from remote peer f5bf427196554e70b62d858f83ce80dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:16:04.769673 4956 consensus_queue.cc:1035] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4671291604d3438e90e5bbdc94701fae" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 44223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:16:04.770038 4952 consensus_queue.cc:1035] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "35d0121d93d94817924e71f0d0de5ceb" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:16:04.782508 4963 mvcc.cc:205] Tried to move back new op lower bound from 7317448969279692800 to 7317448968889098240. Current Snapshot: MvccSnapshot[applied={T|T < 7317448969279692800}]
I20260811 22:16:04.799746 4206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:16:04.800079 4206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:16:04.803642 4206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Sending DeleteTablet for 3 replicas of tablet b2d434b1dd904b959efe0d73d15f4382
I20260811 22:16:04.804961 4613 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d434b1dd904b959efe0d73d15f4382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:04 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:47196
I20260811 22:16:04.805027 4399 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d434b1dd904b959efe0d73d15f4382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:04 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:46632
I20260811 22:16:04.805634 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 4264
I20260811 22:16:04.805811 4824 tablet_service.cc:1514] Processing DeleteTablet for tablet b2d434b1dd904b959efe0d73d15f4382 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:04 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:56250
I20260811 22:16:04.806784 4979 tablet_replica.cc:330] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: stopping tablet replica
I20260811 22:16:04.807228 4979 raft_consensus.cc:2229] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:16:04.808082 4979 raft_consensus.cc:2258] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:04.808900 4981 tablet_replica.cc:330] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: stopping tablet replica
I20260811 22:16:04.809327 4981 raft_consensus.cc:2229] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:16:04.809881 4981 raft_consensus.cc:2258] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:04.812081 4979 ts_tablet_manager.cc:1905] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:04.818279 4980 tablet_replica.cc:330] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: stopping tablet replica
I20260811 22:16:04.818900 4980 raft_consensus.cc:2229] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:16:04.819550 4980 raft_consensus.cc:2258] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:04.820035 4979 ts_tablet_manager.cc:1918] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:04.820230 4979 log.cc:1196] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/b2d434b1dd904b959efe0d73d15f4382
I20260811 22:16:04.820879 4979 ts_tablet_manager.cc:1939] T b2d434b1dd904b959efe0d73d15f4382 P f5bf427196554e70b62d858f83ce80dc: Deleting consensus metadata
I20260811 22:16:04.822290 4980 ts_tablet_manager.cc:1905] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:04.824347 4981 ts_tablet_manager.cc:1905] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:04.827916 4934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.806717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.10.173.129:44223)
0811 22:16:04.806925 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.806933 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.807425 (+ 492us) server_negotiation.cc:407] Connection header received
0811 22:16:04.808111 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.808118 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.808206 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.808286 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:04.811168 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.812387 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.814518 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.815218 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.815316 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.819253 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:04.819271 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:04.819289 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:04.819331 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:04.823689 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:04.824869 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:04.824880 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:04.824884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:04.825210 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:04.825743 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:04.825749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:04.825754 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:04.826225 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:04.826576 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.827224 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.827505 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:16:04.829726 4499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:04.832726 4981 ts_tablet_manager.cc:1918] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:04.832964 4981 log.cc:1196] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/b2d434b1dd904b959efe0d73d15f4382
I20260811 22:16:04.833480 4981 ts_tablet_manager.cc:1939] T b2d434b1dd904b959efe0d73d15f4382 P 35d0121d93d94817924e71f0d0de5ceb: Deleting consensus metadata
I20260811 22:16:04.835086 4190 catalog_manager.cc:4841] TS f5bf427196554e70b62d858f83ce80dc (127.10.173.130:34559): tablet b2d434b1dd904b959efe0d73d15f4382 (table test-table [id=24799d4a947c4ace96fe5f8295a6f255]) successfully deleted
I20260811 22:16:04.835987 4189 catalog_manager.cc:4841] TS 35d0121d93d94817924e71f0d0de5ceb (127.10.173.131:43901): tablet b2d434b1dd904b959efe0d73d15f4382 (table test-table [id=24799d4a947c4ace96fe5f8295a6f255]) successfully deleted
I20260811 22:16:04.926487 4980 ts_tablet_manager.cc:1918] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:04.926753 4980 log.cc:1196] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/b2d434b1dd904b959efe0d73d15f4382
I20260811 22:16:04.927196 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 4264
W20260811 22:16:04.961795 4190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.173.129:44223 (error 108)
I20260811 22:16:04.962018 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 4511
I20260811 22:16:04.964200 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.963371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:46650)
0811 22:16:04.963816 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:04.963973 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":207,"threads_started":1}
W20260811 22:16:04.964509 4190 catalog_manager.cc:4584] TS 4671291604d3438e90e5bbdc94701fae (127.10.173.129:44223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b2d434b1dd904b959efe0d73d15f4382: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
I20260811 22:16:04.979125 4941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:04.963071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47208 (local address 127.10.173.130:34559)
0811 22:16:04.963325 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:04.963332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:04.963352 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:16:04.963498 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:04.963505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:04.963606 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:04.963713 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:04.965831 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.966776 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.968647 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:04.969091 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:04.969209 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:04.972763 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:04.972777 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:04.972793 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:04.972838 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:04.976027 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:04.976908 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:04.976915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:04.976917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:04.977053 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:04.977487 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:04.977492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:04.977494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:04.977873 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:04.978188 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:04.978586 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 22:16:04.978760 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:16:04.979861 4706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:05.018461 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:05.017862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:46652)
0811 22:16:05.018127 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:05.018254 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 22:16:05.019709 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:05.019232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:46668)
0811 22:16:05.019426 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:05.019521 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 22:16:05.080252 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:05.079677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:46680)
0811 22:16:05.079944 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:05.080065 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 22:16:05.081420 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:05.080963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:46684)
0811 22:16:05.081148 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:05.081247 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 22:16:05.101030 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 4511
I20260811 22:16:05.128099 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 4717
I20260811 22:16:05.146360 4938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:05.128934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56252 (local address 127.10.173.131:43901)
0811 22:16:05.129161 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:05.129171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:05.129190 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:16:05.129362 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:05.129366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:05.129433 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:05.129538 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:05.131153 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:05.131945 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:05.133458 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:05.133852 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:05.133946 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:05.140548 (+ 6602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:05.140562 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:05.140580 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:05.140623 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:05.143663 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:05.144368 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:05.144376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:05.144378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:05.144530 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:05.144892 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:05.144895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:05.144897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:05.145311 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:05.145533 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:05.145887 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 22:16:05.146056 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:16:05.147105 4905 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:05.162509 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:05.161871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:46700)
0811 22:16:05.162149 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:05.162285 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 22:16:05.163688 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:05.163185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:46716)
0811 22:16:05.163439 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:05.163528 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260811 22:16:05.237071 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 4717
I20260811 22:16:05.265460 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 4158
I20260811 22:16:05.266896 4253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:05.380971 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 4158
I20260811 22:16:05.381057 4985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:05.379968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:44223 (local address 127.0.0.1:46720)
0811 22:16:05.380326 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:05.380443 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
I20260811 22:16:25.413659 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:42607
--webserver_interface=127.10.173.190
--webserver_port=36385
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:42607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:25.769251 4992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:25.769534 4992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:25.769701 4992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:25.777542 4992 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:16:25.777637 4992 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:16:25.777725 4992 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:16:25.777832 4992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:16:25.777931 4992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:25.778082 4992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:25.778138 4992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:16:25.778211 4992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:16:25.787919 4992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:42607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:42607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=36385
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:25.789290 4992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:25.790838 4992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:25.803762 5003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:25.803833 5000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:25.803854 4999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:25.804647 4992 server_base.cc:1034] running on GCE node
I20260811 22:16:25.805490 4992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:25.806998 4992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:25.808287 4992 hybrid_clock.cc:648] HybridClock initialized: now 1786486585808203 us; error 48 us; skew 500 ppm
I20260811 22:16:25.808650 4992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:25 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486585, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:25.819413 4992 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:16:25.821087 4992 webserver.cc:460] Webserver started at http://127.10.173.190:36385/ using document root <none> and password file <none>
I20260811 22:16:25.821825 4992 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:25.821919 4992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:25.827764 4992 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:16:25.830698 5014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:25.832026 4992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:16:25.832166 4992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "9069c916037c45fbb1be6e67ddcadbed"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-68xk"
server_key: "10b6c29efe77ea585c4242ffcf58bba6"
server_key_iv: "4ae2f1e31f404f83b5cf322cf7d87a85"
server_key_version: "encryptionkey@0"
I20260811 22:16:25.833109 4992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:25.855913 4992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:25.860474 4992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:25.860754 4992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:25.881520 4992 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:42607
I20260811 22:16:25.881539 5092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:42607 every 8 connection(s)
I20260811 22:16:25.883294 4992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:16:25.886569 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 4992
I20260811 22:16:25.887568 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:44223
--local_ip_for_outbound_sockets=127.10.173.129
--tserver_master_addrs=127.10.173.190:42607
--webserver_port=33441
--webserver_interface=127.10.173.129
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:16:25.892867 5093 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:16:25.896394 5093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Bootstrap starting.
I20260811 22:16:25.926141 5093 log.cc:824] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:25.947762 5093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:16:25.948364 5093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Bootstrap complete.
I20260811 22:16:25.957641 5093 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:16:25.958612 5093 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9069c916037c45fbb1be6e67ddcadbed, State: Initialized, Role: FOLLOWER
I20260811 22:16:25.959247 5093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:16:25.959491 5093 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:16:25.959607 5093 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:16:25.959803 5093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:16:25.963869 5093 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:16:25.964432 5093 leader_election.cc:302] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9069c916037c45fbb1be6e67ddcadbed; no voters:
I20260811 22:16:25.965155 5093 leader_election.cc:288] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:16:25.965303 5097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:16:25.966746 5097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [term 3 LEADER]: Becoming Leader. State: Replica: 9069c916037c45fbb1be6e67ddcadbed, State: Running, Role: LEADER
I20260811 22:16:25.967337 5097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } }
I20260811 22:16:25.967619 5093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:16:25.970274 5100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9069c916037c45fbb1be6e67ddcadbed. Latest consensus state: current_term: 3 leader_uuid: "9069c916037c45fbb1be6e67ddcadbed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } } }
I20260811 22:16:25.970726 5100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:25.971347 5099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9069c916037c45fbb1be6e67ddcadbed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9069c916037c45fbb1be6e67ddcadbed" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42607 } } }
I20260811 22:16:25.971673 5099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:25.976436 5108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:16:25.987411 5108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:16:25.989391 5108 catalog_manager.cc:1259] Loaded cluster ID: 955e4bc07f5e4550a2b1a152a939831c
I20260811 22:16:25.989519 5108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:16:25.995406 5108 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:16:25.997526 5108 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Loaded TSK: 0
I20260811 22:16:25.997887 5108 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9069c916037c45fbb1be6e67ddcadbed: Loaded TSK: 1
F20260811 22:16:25.999434 5108 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f94921f49ad google::LogMessage::Fail() at ??:0
@ 0x7f94921f788a google::LogMessage::SendToLog() at ??:0
@ 0x7f94921f44de google::LogMessage::Flush() at ??:0
@ 0x7f94921f5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9495f587d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9495f4773e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f94a7c231c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f94a7c22783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f94a7cae2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f94a7cae0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f94a7cf85e9 std::function<>::operator()() at ??:0
@ 0x7f94a7c26cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f94a7c24c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f94a7cac610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f94a7cac439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9494e40345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9494e4391c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f9494e43799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9494e1eeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f94980f86db start_thread at ??:0
@ 0x7f949108e61f clone at ??:0
W20260811 22:16:26.346549 5095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:26.346836 5095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:26.347020 5095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:26.354663 5095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:26.354789 5095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:26.355001 5095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:16:26.365235 5095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:44223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=33441
--tserver_master_addrs=127.10.173.190:42607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:26.366652 5095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:26.368060 5095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:26.381945 5152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:26.381909 5157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:26.381867 5154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:26.383155 5095 server_base.cc:1034] running on GCE node
I20260811 22:16:26.383761 5095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:26.384687 5095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:26.385895 5095 hybrid_clock.cc:648] HybridClock initialized: now 1786486586385848 us; error 46 us; skew 500 ppm
I20260811 22:16:26.386241 5095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:26 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486586, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:26.389958 5095 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:16:26.391746 5095 webserver.cc:460] Webserver started at http://127.10.173.129:33441/ using document root <none> and password file <none>
I20260811 22:16:26.392467 5095 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:26.392588 5095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:26.398638 5095 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:16:26.401716 5166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:26.403256 5095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:16:26.403460 5095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "4671291604d3438e90e5bbdc94701fae"
format_stamp: "Formatted at 2026-08-11 22:15:39 on dist-test-slave-68xk"
server_key: "e4b6e6049feae08743a5eea58606afcc"
server_key_iv: "6be02ce6fbd28113391002a10b0f344e"
server_key_version: "encryptionkey@0"
I20260811 22:16:26.404597 5095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:26.422962 5095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:26.427716 5095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:26.428042 5095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:26.429131 5095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:26.431214 5178 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 22:16:26.437711 5178 ts_tablet_manager.cc:1852] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 22:16:26.438793 5178 ts_tablet_manager.cc:1905] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:26.446627 5178 ts_tablet_manager.cc:1918] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:26.446949 5178 ts_tablet_manager.cc:1939] T b2d434b1dd904b959efe0d73d15f4382 P 4671291604d3438e90e5bbdc94701fae: Deleting consensus metadata
I20260811 22:16:26.447575 5095 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 22:16:26.447727 5095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.017s user 0.001s sys 0.000s
I20260811 22:16:26.447883 5095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:26.447994 5095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:26.484766 5095 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:44223
I20260811 22:16:26.484865 5336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:44223 every 8 connection(s)
I20260811 22:16:26.486119 5095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:16:26.491277 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 5095
I20260811 22:16:26.491961 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:34559
--local_ip_for_outbound_sockets=127.10.173.130
--tserver_master_addrs=127.10.173.190:42607
--webserver_port=36445
--webserver_interface=127.10.173.130
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:16:26.492445 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:26.489580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:49435)
0811 22:16:26.491192 (+ 1612us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:26.491490 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1273,"thread_start_us":1074,"threads_started":1}
W20260811 22:16:26.493021 5337 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:42607 (0 consecutive failures): Network error: Failed to ping master at 127.10.173.190:42607: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
I20260811 22:16:26.493897 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:26.493592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:38617)
0811 22:16:26.493733 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:26.493799 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:16:26.494820 5339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:26.494520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:54173)
0811 22:16:26.494631 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:26.494703 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 22:16:26.495066 5337 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:16:26.834550 5340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:26.834861 5340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:26.834980 5340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:26.842157 5340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:26.842267 5340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:26.842382 5340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:16:26.852581 5340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:34559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=36445
--tserver_master_addrs=127.10.173.190:42607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:26.853896 5340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:26.855163 5340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:26.869287 5349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:26.869199 5353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:26.869226 5350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:26.872025 5340 server_base.cc:1034] running on GCE node
I20260811 22:16:26.872747 5340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:26.873764 5340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:26.874979 5340 hybrid_clock.cc:648] HybridClock initialized: now 1786486586874918 us; error 53 us; skew 500 ppm
I20260811 22:16:26.875325 5340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:26 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486586, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:26.878904 5340 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:16:26.880659 5340 webserver.cc:460] Webserver started at http://127.10.173.130:36445/ using document root <none> and password file <none>
I20260811 22:16:26.881443 5340 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:26.881567 5340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:26.887282 5340 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:16:26.890098 5364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:26.891923 5340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:16:26.892127 5340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "f5bf427196554e70b62d858f83ce80dc"
format_stamp: "Formatted at 2026-08-11 22:15:40 on dist-test-slave-68xk"
server_key: "726fd98158e559610a0f3076441850d7"
server_key_iv: "f6bbe2e6669ed71962d86a5b3fc68527"
server_key_version: "encryptionkey@0"
I20260811 22:16:26.893168 5340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:26.923372 5340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:26.928004 5340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:26.928341 5340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:26.929518 5340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:26.931047 5340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:26.931140 5340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:26.931305 5340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:26.931402 5340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:26.966709 5340 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:34559
I20260811 22:16:26.966823 5518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:34559 every 8 connection(s)
I20260811 22:16:26.968173 5340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:16:26.972903 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 5340
I20260811 22:16:26.973470 5522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:26.971266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:47049)
0811 22:16:26.972017 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:26.972273 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":310,"threads_started":1}
I20260811 22:16:26.973730 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:43901
--local_ip_for_outbound_sockets=127.10.173.131
--tserver_master_addrs=127.10.173.190:42607
--webserver_port=42121
--webserver_interface=127.10.173.131
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:26.974387 5520 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:42607 (0 consecutive failures): Network error: Failed to ping master at 127.10.173.190:42607: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
I20260811 22:16:26.975466 5522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:26.975006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:34289)
0811 22:16:26.975203 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:26.975304 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:16:26.976511 5522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:26.976089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:44175)
0811 22:16:26.976258 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:26.976356 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:16:26.976814 5520 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:16:27.331494 5523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:27.331809 5523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:27.331975 5523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:27.339432 5523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:27.339557 5523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:27.339748 5523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:16:27.349799 5523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:43901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=42121
--tserver_master_addrs=127.10.173.190:42607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:27.351239 5523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:27.352586 5523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:27.366233 5532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:27.366283 5529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:27.366317 5530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:27.367657 5523 server_base.cc:1034] running on GCE node
I20260811 22:16:27.368335 5523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:27.369288 5523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:27.370532 5523 hybrid_clock.cc:648] HybridClock initialized: now 1786486587370487 us; error 36 us; skew 500 ppm
I20260811 22:16:27.370860 5523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:27 dist-test-slave-68xk krb5kdc[3314](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486587, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:27.374342 5523 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:16:27.375841 5523 webserver.cc:460] Webserver started at http://127.10.173.131:42121/ using document root <none> and password file <none>
I20260811 22:16:27.376581 5523 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:27.376718 5523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:27.381944 5523 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:16:27.384780 5541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:27.386241 5523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:16:27.386467 5523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "35d0121d93d94817924e71f0d0de5ceb"
format_stamp: "Formatted at 2026-08-11 22:15:41 on dist-test-slave-68xk"
server_key: "d9d5ab67dbcaa11f7850bcb114f1c80d"
server_key_iv: "b8c31405824c64ae87d5fdfe17702afa"
server_key_version: "encryptionkey@0"
I20260811 22:16:27.387580 5523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:27.412143 5523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:27.416530 5523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:27.416833 5523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:27.417922 5523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:27.419425 5523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:27.419519 5523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:27.419662 5523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:27.419762 5523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:27.456115 5523 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:43901
I20260811 22:16:27.456260 5698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:43901 every 8 connection(s)
I20260811 22:16:27.457604 5523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:16:27.458469 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 5523
I20260811 22:16:27.462996 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:27.461044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:58281)
0811 22:16:27.461796 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:27.462082 (+ 286us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":297,"threads_started":1}
W20260811 22:16:27.463789 5700 heartbeater.cc:644] Failed to heartbeat to 127.10.173.190:42607 (0 consecutive failures): Network error: Failed to ping master at 127.10.173.190:42607: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
I20260811 22:16:27.464665 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:27.464323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:39557)
0811 22:16:27.464480 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:27.464555 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 22:16:27.465559 5704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:27.465249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:52041)
0811 22:16:27.465379 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:27.465459 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 22:16:27.465849 5700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:16:27.496678 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:27.495802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:38525)
0811 22:16:27.496338 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:27.496422 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":256,"threads_started":1}
I20260811 22:16:27.979300 5707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:27.977696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:51521)
0811 22:16:27.978579 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:27.978726 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":486,"threads_started":1}
I20260811 22:16:28.468072 5708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:28.466783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:33923)
0811 22:16:28.467485 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:28.467644 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":303,"threads_started":1}
I20260811 22:16:28.500306 5709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:28.498334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:39361)
0811 22:16:28.499732 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:28.499928 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1144,"thread_start_us":777,"threads_started":1}
I20260811 22:16:28.981201 5710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:28.980421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:38439)
0811 22:16:28.980942 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:28.981025 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":286,"threads_started":1}
I20260811 22:16:29.470288 5711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:29.469366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:40533)
0811 22:16:29.469985 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:29.470099 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":330,"threads_started":1}
I20260811 22:16:29.502548 5712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:29.501536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:56011)
0811 22:16:29.502160 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:29.502293 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":308,"threads_started":1}
I20260811 22:16:29.982964 5714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:29.982125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:39403)
0811 22:16:29.982693 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:29.982777 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":323,"threads_started":1}
I20260811 22:16:30.472136 5715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:30.471271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:38977)
0811 22:16:30.471846 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:30.471936 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":335,"threads_started":1}
I20260811 22:16:30.504558 5716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:30.503705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:55949)
0811 22:16:30.504242 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:30.504336 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":280,"threads_started":1}
I20260811 22:16:30.984694 5717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:30.983940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:59759)
0811 22:16:30.984403 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:30.984487 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":243,"threads_started":1}
I20260811 22:16:31.475131 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:31.474020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:58847)
0811 22:16:31.474871 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:31.474957 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":606,"threads_started":1}
I20260811 22:16:31.506733 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:31.505792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:56657)
0811 22:16:31.506308 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:31.506399 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":252,"threads_started":1}
I20260811 22:16:31.986611 5721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:31.985713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:38405)
0811 22:16:31.986265 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:31.986385 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":249,"threads_started":1}
I20260811 22:16:32.476930 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:32.476093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:46197)
0811 22:16:32.476652 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:32.476755 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":320,"threads_started":1}
I20260811 22:16:32.508539 5726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:32.507708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:35013)
0811 22:16:32.508170 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:32.508270 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":251,"threads_started":1}
I20260811 22:16:32.988741 5727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:32.987776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:45009)
0811 22:16:32.988430 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:32.988542 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":387,"threads_started":1}
I20260811 22:16:33.478823 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:33.477837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:43283)
0811 22:16:33.478509 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:33.478636 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":355,"threads_started":1}
I20260811 22:16:33.510697 5729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:33.509636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:44291)
0811 22:16:33.510265 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:33.510381 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":257,"threads_started":1}
I20260811 22:16:33.991047 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:33.989773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:45173)
0811 22:16:33.990673 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:33.990815 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":516,"threads_started":1}
I20260811 22:16:34.480814 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:34.479807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:46305)
0811 22:16:34.480361 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:34.480520 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":265,"threads_started":1}
I20260811 22:16:34.512472 5735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:34.511624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:49255)
0811 22:16:34.512169 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:34.512263 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":268,"threads_started":1}
I20260811 22:16:34.992928 5737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:34.992070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:40053)
0811 22:16:34.992658 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:34.992760 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":331,"threads_started":1}
I20260811 22:16:35.482761 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:35.481852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:44057)
0811 22:16:35.482394 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:35.482583 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":321,"threads_started":1}
I20260811 22:16:35.514318 5744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:35.513450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:40099)
0811 22:16:35.513975 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:35.514128 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":266,"threads_started":1}
I20260811 22:16:35.994954 5745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:35.994058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:48607)
0811 22:16:35.994669 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:35.994766 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":277,"threads_started":1}
I20260811 22:16:36.484877 5746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:36.483903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:47579)
0811 22:16:36.484608 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:36.484694 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":432,"threads_started":1}
I20260811 22:16:36.516363 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:36.515462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:58697)
0811 22:16:36.516074 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:36.516163 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":327,"threads_started":1}
I20260811 22:16:36.996719 5748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:36.995945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:37941)
0811 22:16:36.996440 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:36.996539 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":265,"threads_started":1}
I20260811 22:16:37.486820 5749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:37.486026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:42799)
0811 22:16:37.486577 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:37.486664 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":285,"threads_started":1}
I20260811 22:16:37.518311 5750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:37.517429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:47653)
0811 22:16:37.518034 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:37.518126 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":381,"threads_started":1}
I20260811 22:16:37.998479 5752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:37.997706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:39347)
0811 22:16:37.998134 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:37.998234 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":213,"threads_started":1}
I20260811 22:16:38.488734 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:38.487815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:35879)
0811 22:16:38.488429 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:38.488544 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":367,"threads_started":1}
I20260811 22:16:38.520035 5755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:38.519272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:47153)
0811 22:16:38.519775 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:38.519860 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":279,"threads_started":1}
I20260811 22:16:39.000214 5756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:38.999417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:53227)
0811 22:16:38.999914 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:39.000000 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":275,"threads_started":1}
I20260811 22:16:39.490475 5757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:39.489669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:41231)
0811 22:16:39.490154 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:39.490246 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":246,"threads_started":1}
I20260811 22:16:39.521904 5758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:39.521128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:47709)
0811 22:16:39.521624 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:39.521729 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":263,"threads_started":1}
I20260811 22:16:40.002128 5760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:40.001225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:51221)
0811 22:16:40.001807 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:40.001917 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":279,"threads_started":1}
I20260811 22:16:40.492262 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:40.491432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:48443)
0811 22:16:40.491979 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:40.492076 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":276,"threads_started":1}
I20260811 22:16:40.523640 5762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:40.522804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:32965)
0811 22:16:40.523337 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:40.523428 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":257,"threads_started":1}
I20260811 22:16:41.004060 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:41.003316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:58025)
0811 22:16:41.003783 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:41.003869 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":231,"threads_started":1}
I20260811 22:16:41.494019 5764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:41.493229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:37283)
0811 22:16:41.493738 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:41.493828 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":260,"threads_started":1}
I20260811 22:16:41.525923 5765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:41.525020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:53023)
0811 22:16:41.525499 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:41.525664 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":206,"threads_started":1}
I20260811 22:16:42.006281 5766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:42.005390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.130:34419)
0811 22:16:42.005915 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:42.006044 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":288,"threads_started":1}
I20260811 22:16:42.466377 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 5095
I20260811 22:16:42.485913 5770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:42.467936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.10.173.129:44223)
0811 22:16:42.468695 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:42.468710 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:42.468750 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:16:42.470567 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:42.470579 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:42.471022 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:42.471406 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:42.472976 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:42.474296 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:42.475671 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:42.476353 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:42.476572 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:42.479500 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:42.479508 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:42.479524 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:42.479561 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:42.482523 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:42.483226 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:42.483237 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:42.483250 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:42.483414 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:42.483829 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:42.483835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:42.483838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:42.484379 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:42.484711 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:42.485162 (+ 451us) server_negotiation.cc:299] Negotiation successful
0811 22:16:42.485601 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":385,"threads_started":1}
I20260811 22:16:42.487576 5329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:42.496161 5772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:42.495348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.131:37195)
0811 22:16:42.495824 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:42.495940 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":247,"threads_started":1}
I20260811 22:16:42.578493 5773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:42.577149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:42607 (local address 127.10.173.129:48269)
0811 22:16:42.577847 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:42.578104 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.190:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":326,"threads_started":1}
I20260811 22:16:42.578447 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 5095
I20260811 22:16:42.604904 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 5340
I20260811 22:16:42.624176 5776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:42.606039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39524 (local address 127.10.173.130:34559)
0811 22:16:42.606739 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:42.606754 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:42.606797 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:16:42.608376 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:42.608386 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:42.608847 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:42.609158 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:42.610822 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:42.612133 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:42.613450 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:42.614190 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:42.614332 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:42.617272 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:42.617279 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:42.617296 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:42.617336 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:42.620499 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:42.621300 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:42.621306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:42.621313 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:42.621491 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:42.621962 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:42.621966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:42.621968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:42.622711 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:42.622950 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:42.623394 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 22:16:42.623865 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":392,"threads_started":1}
I20260811 22:16:42.625746 5512 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:42.712252 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 5340
I20260811 22:16:42.737947 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 5523
I20260811 22:16:42.757485 5778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:42.739059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38180 (local address 127.10.173.131:43901)
0811 22:16:42.739638 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:42.739654 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:42.739696 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:16:42.741362 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:42.741372 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:42.741801 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:42.742170 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:42.743814 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:42.745054 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:42.746356 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:42.747085 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:42.747308 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:42.750281 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:42.750289 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:42.750304 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:42.750342 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:42.753553 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:42.754578 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:42.754587 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:42.754594 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:42.754780 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:42.755241 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:42.755248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:42.755251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:42.755812 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:42.756171 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:42.756703 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 22:16:42.757153 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":262,"threads_started":1}
I20260811 22:16:42.759133 5693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:42.843487 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 5523
2026-08-11T22:16:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65639 ms)
[----------] 30 tests from SecurityITest (317516 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:16:42 dist-test-slave-68xk krb5kdc[5782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:16:42 dist-test-slave-68xk krb5kdc[5782](info): set up 2 sockets
Aug 11 22:16:42 dist-test-slave-68xk krb5kdc[5782](info): commencing operation
krb5kdc: starting...
W20260811 22:16:44.300817 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.408s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:16:44 dist-test-slave-68xk krb5kdc[5782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:16:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:16:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:16:44.627416 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:46825
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:46825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:44.979405 5797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:44.979795 5797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:44.979921 5797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:44.987341 5797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:16:44.987449 5797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:44.987496 5797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:44.987553 5797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:16:44.987598 5797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:16:44.997705 5797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:46825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:46825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:44.999125 5797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:45.000439 5797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:45.015465 5802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:45.015787 5804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:45.016535 5807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:45.016578 5797 server_base.cc:1034] running on GCE node
I20260811 22:16:45.017483 5797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:45.019089 5797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:45.020346 5797 hybrid_clock.cc:648] HybridClock initialized: now 1786486605020301 us; error 37 us; skew 500 ppm
I20260811 22:16:45.020717 5797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:45 dist-test-slave-68xk krb5kdc[5782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486605, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:45.024041 5797 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:16:45.025720 5797 webserver.cc:460] Webserver started at http://127.10.173.190:36391/ using document root <none> and password file <none>
I20260811 22:16:45.026513 5797 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:45.026680 5797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:45.027012 5797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:45.030035 5797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "e0b6d1cca81a419cace997e49ee0e4ef"
format_stamp: "Formatted at 2026-08-11 22:16:45 on dist-test-slave-68xk"
server_key: "899a309e5451afedea5e2e20e760d66b"
server_key_iv: "0429c35c92fd7baa1dc971226980307c"
server_key_version: "encryptionkey@0"
I20260811 22:16:45.030856 5797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "e0b6d1cca81a419cace997e49ee0e4ef"
format_stamp: "Formatted at 2026-08-11 22:16:45 on dist-test-slave-68xk"
server_key: "899a309e5451afedea5e2e20e760d66b"
server_key_iv: "0429c35c92fd7baa1dc971226980307c"
server_key_version: "encryptionkey@0"
I20260811 22:16:45.036116 5797 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:16:45.039747 5819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:45.041167 5797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:16:45.041392 5797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "e0b6d1cca81a419cace997e49ee0e4ef"
format_stamp: "Formatted at 2026-08-11 22:16:45 on dist-test-slave-68xk"
server_key: "899a309e5451afedea5e2e20e760d66b"
server_key_iv: "0429c35c92fd7baa1dc971226980307c"
server_key_version: "encryptionkey@0"
I20260811 22:16:45.041602 5797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:45.071403 5797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:45.076135 5797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:45.076468 5797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:45.096269 5797 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:46825
I20260811 22:16:45.096309 5889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:46825 every 8 connection(s)
I20260811 22:16:45.097746 5797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:16:45.099077 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 5797
I20260811 22:16:45.099339 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:16:45.100199 10934 external_mini_cluster.cc:1428] Setting key a3b01ab47e7b85c7c074040acd4afc41
I20260811 22:16:45.102865 5890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 22:16:45 dist-test-slave-68xk krb5kdc[5782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:45.116400 5890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef: Bootstrap starting.
I20260811 22:16:45.119656 5890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:45.120748 5890 log.cc:824] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:45.123754 5890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef: No bootstrap required, opened a new log
I20260811 22:16:45.124598 5895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:45.102338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.10.173.190:46825)
0811 22:16:45.103227 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:45.103238 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:45.103283 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:16:45.104898 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:45.104919 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:45.105319 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:45.105641 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:45.107630 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:45.108977 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:45.110967 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:45.111692 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:45.111831 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:45.116894 (+ 5063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:45.116901 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:45.116915 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:45.116952 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:45.120303 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:45.121069 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:45.121079 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:45.121089 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:45.121270 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:45.121727 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:45.121735 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:45.121738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:45.122266 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:45.122615 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:45.123094 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 22:16:45.123528 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":381,"threads_started":1}
I20260811 22:16:45.131559 5890 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b6d1cca81a419cace997e49ee0e4ef" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46825 } }
I20260811 22:16:45.132017 5890 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:45.132134 5890 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0b6d1cca81a419cace997e49ee0e4ef, State: Initialized, Role: FOLLOWER
I20260811 22:16:45.132712 5890 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b6d1cca81a419cace997e49ee0e4ef" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46825 } }
I20260811 22:16:45.132982 5890 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:16:45.133136 5890 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:16:45.133337 5890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:45.135751 5890 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b6d1cca81a419cace997e49ee0e4ef" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46825 } }
I20260811 22:16:45.136214 5890 leader_election.cc:302] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0b6d1cca81a419cace997e49ee0e4ef; no voters:
I20260811 22:16:45.136895 5890 leader_election.cc:288] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:16:45.137028 5897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:16:45.137778 5897 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [term 1 LEADER]: Becoming Leader. State: Replica: e0b6d1cca81a419cace997e49ee0e4ef, State: Running, Role: LEADER
I20260811 22:16:45.138281 5897 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b6d1cca81a419cace997e49ee0e4ef" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46825 } }
I20260811 22:16:45.139101 5890 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:16:45.143064 5898 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0b6d1cca81a419cace997e49ee0e4ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b6d1cca81a419cace997e49ee0e4ef" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46825 } } }
I20260811 22:16:45.143790 5900 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0b6d1cca81a419cace997e49ee0e4ef. Latest consensus state: current_term: 1 leader_uuid: "e0b6d1cca81a419cace997e49ee0e4ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0b6d1cca81a419cace997e49ee0e4ef" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 46825 } } }
I20260811 22:16:45.144171 5898 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:45.144217 5900 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:45.145210 5909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:16:45.150444 5909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:16:45.159088 5909 catalog_manager.cc:1347] Generated new cluster ID: 9fb4b5f13fa541e786d922dbd2108bd7
I20260811 22:16:45.159212 5909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:16:45.183918 5909 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:16:45.185380 5909 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:16:45.198913 5909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef: Generated new TSK 0
I20260811 22:16:45.199909 5909 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:16:45.283121 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46825
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:45.655704 5930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:45.655992 5930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:45.656090 5930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:45.664795 5930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:45.664911 5930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:45.665028 5930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:16:45.674926 5930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:45.676174 5930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:45.677467 5930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:45.691048 5936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:45.691203 5937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:45.691406 5941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:45.692432 5930 server_base.cc:1034] running on GCE node
I20260811 22:16:45.693117 5930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:45.694118 5930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:45.695369 5930 hybrid_clock.cc:648] HybridClock initialized: now 1786486605695306 us; error 60 us; skew 500 ppm
I20260811 22:16:45.695708 5930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:45 dist-test-slave-68xk krb5kdc[5782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486605, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:45.698966 5930 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:16:45.700508 5930 webserver.cc:460] Webserver started at http://127.10.173.129:33443/ using document root <none> and password file <none>
I20260811 22:16:45.701238 5930 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:45.701395 5930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:45.701731 5930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:45.704741 5930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "96c0db61e19e4a4ca5b8331e3e5f8920"
format_stamp: "Formatted at 2026-08-11 22:16:45 on dist-test-slave-68xk"
server_key: "54367fdab861ed0d05759c54628eab36"
server_key_iv: "e3f029ce4e32c9e26fe7abde77b84594"
server_key_version: "encryptionkey@0"
I20260811 22:16:45.705564 5930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "96c0db61e19e4a4ca5b8331e3e5f8920"
format_stamp: "Formatted at 2026-08-11 22:16:45 on dist-test-slave-68xk"
server_key: "54367fdab861ed0d05759c54628eab36"
server_key_iv: "e3f029ce4e32c9e26fe7abde77b84594"
server_key_version: "encryptionkey@0"
I20260811 22:16:45.710875 5930 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 22:16:45.714313 5948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:45.715837 5930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:16:45.716020 5930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "96c0db61e19e4a4ca5b8331e3e5f8920"
format_stamp: "Formatted at 2026-08-11 22:16:45 on dist-test-slave-68xk"
server_key: "54367fdab861ed0d05759c54628eab36"
server_key_iv: "e3f029ce4e32c9e26fe7abde77b84594"
server_key_version: "encryptionkey@0"
I20260811 22:16:45.716236 5930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:45.742630 5930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:45.747203 5930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:45.747511 5930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:45.748502 5930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:45.749841 5930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:45.749955 5930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:45.750110 5930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:45.750195 5930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:45.784592 5930 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:36429
I20260811 22:16:45.784646 6106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:36429 every 8 connection(s)
I20260811 22:16:45.785940 5930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:16:45.791424 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 5930
I20260811 22:16:45.791651 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:16:45.792302 10934 external_mini_cluster.cc:1428] Setting key 7e1c55f0924bc7272f5fb67e48a4811c
Aug 11 22:16:45 dist-test-slave-68xk krb5kdc[5782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486605, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:45.806841 6109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:45.788632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:37397 (local address 127.10.173.190:46825)
0811 22:16:45.789103 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:45.789107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:45.790042 (+ 935us) server_negotiation.cc:407] Connection header received
0811 22:16:45.792162 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:45.792168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:45.792255 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:45.792377 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:45.794147 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:45.795008 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:45.796626 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:45.797025 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:45.797181 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:45.800874 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:45.800879 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:45.800883 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:45.800918 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:45.803576 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:45.804347 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:45.804353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:45.804357 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:45.804486 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:45.804949 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:45.804957 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:45.804960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:45.805267 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:45.805435 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:45.806200 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 22:16:45.806461 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":228,"threads_started":1}
I20260811 22:16:45.807869 6110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:45.789150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46825 (local address 127.10.173.129:37397)
0811 22:16:45.789853 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:45.789897 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:45.791886 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:45.792614 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:45.792622 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:45.793316 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:45.793888 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:45.793906 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:45.795168 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:45.795174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:45.796437 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:45.796458 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:45.797274 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:45.797284 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:45.797547 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:45.798285 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:16:45.798309 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:45.800622 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:45.803742 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:45.803751 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:45.803769 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:45.804218 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:45.804614 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:45.804619 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:45.804622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:45.804830 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:45.805406 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:45.805414 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:45.805784 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 22:16:45.806072 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 22:16:45.806661 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":268,"threads_started":1}
I20260811 22:16:45.809520 6107 heartbeater.cc:347] Connected to a master server at 127.10.173.190:46825
I20260811 22:16:45.809877 6107 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:45.810691 6107 heartbeater.cc:511] Master 127.10.173.190:46825 requested a full tablet report, sending...
I20260811 22:16:45.812837 5845 ts_manager.cc:194] Registered new tserver with Master: 96c0db61e19e4a4ca5b8331e3e5f8920 (127.10.173.129:36429)
I20260811 22:16:45.814347 5845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:37397
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:16:45.865880 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46825
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:46.213269 6113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:46.213549 6113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:46.213696 6113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:46.221611 6113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:46.221735 6113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:46.221910 6113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:16:46.231957 6113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:46.233321 6113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:46.234771 6113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:46.248983 6119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:46.249070 6121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:46.249114 6125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:46.250039 6113 server_base.cc:1034] running on GCE node
I20260811 22:16:46.250773 6113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:46.251750 6113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:46.252983 6113 hybrid_clock.cc:648] HybridClock initialized: now 1786486606252926 us; error 40 us; skew 500 ppm
I20260811 22:16:46.253444 6113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:46 dist-test-slave-68xk krb5kdc[5782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:46.256973 6113 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:16:46.258736 6113 webserver.cc:460] Webserver started at http://127.10.173.130:38469/ using document root <none> and password file <none>
I20260811 22:16:46.259510 6113 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:46.259670 6113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:46.260035 6113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:46.263379 6113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07"
format_stamp: "Formatted at 2026-08-11 22:16:46 on dist-test-slave-68xk"
server_key: "897b0527e3da5aa34f31e6dc4e66ccb4"
server_key_iv: "280a338fee070bfcc84503de8dc78d1f"
server_key_version: "encryptionkey@0"
I20260811 22:16:46.264242 6113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07"
format_stamp: "Formatted at 2026-08-11 22:16:46 on dist-test-slave-68xk"
server_key: "897b0527e3da5aa34f31e6dc4e66ccb4"
server_key_iv: "280a338fee070bfcc84503de8dc78d1f"
server_key_version: "encryptionkey@0"
I20260811 22:16:46.269984 6113 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 22:16:46.273931 6137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:46.275514 6113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:16:46.275735 6113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07"
format_stamp: "Formatted at 2026-08-11 22:16:46 on dist-test-slave-68xk"
server_key: "897b0527e3da5aa34f31e6dc4e66ccb4"
server_key_iv: "280a338fee070bfcc84503de8dc78d1f"
server_key_version: "encryptionkey@0"
I20260811 22:16:46.275967 6113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:46.296967 6113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:46.301270 6113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:46.301594 6113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:46.302706 6113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:46.304152 6113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:46.304260 6113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:46.304417 6113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:46.304493 6113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:46.341552 6113 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:36873
I20260811 22:16:46.341646 6299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:36873 every 8 connection(s)
I20260811 22:16:46.342906 6113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:16:46.348222 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 6113
I20260811 22:16:46.348462 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:16:46.349119 10934 external_mini_cluster.cc:1428] Setting key a3512f0dc9f07089651bccf6644ce69e
Aug 11 22:16:46 dist-test-slave-68xk krb5kdc[5782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:46.370041 6304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.346150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:52839 (local address 127.10.173.190:46825)
0811 22:16:46.346776 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:46.346783 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:46.348151 (+ 1368us) server_negotiation.cc:407] Connection header received
0811 22:16:46.350802 (+ 2651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:46.350807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:46.350873 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:46.350985 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:46.353273 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:46.354008 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:46.356874 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:46.357355 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:46.357468 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:46.362708 (+ 5240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:46.362718 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:46.362727 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:46.362782 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:46.366580 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:46.367398 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:46.367404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:46.367407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:46.367553 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:46.368162 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:46.368166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:46.368168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:46.368492 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:46.368618 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:46.369585 (+ 967us) server_negotiation.cc:299] Negotiation successful
0811 22:16:46.369769 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":332,"threads_started":1}
I20260811 22:16:46.371644 6305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.346891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46825 (local address 127.10.173.130:52839)
0811 22:16:46.347939 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:46.347996 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:46.350465 (+ 2469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:46.351300 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:46.351309 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:46.352144 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:46.353037 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:46.353068 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:46.354198 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:46.354207 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:46.356671 (+ 2464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:46.356691 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:46.357619 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:46.357631 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:46.358020 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:46.358966 (+ 946us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:16:46.359010 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:46.362377 (+ 3367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:46.366762 (+ 4385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:46.366771 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:46.366798 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:46.367251 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:46.367676 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:46.367683 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:46.367689 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:46.367966 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:46.368640 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:46.368650 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:46.369130 (+ 480us) client_negotiation.cc:769] Sending connection context
0811 22:16:46.369479 (+ 349us) client_negotiation.cc:240] Negotiation successful
0811 22:16:46.370069 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":348,"threads_started":1}
I20260811 22:16:46.373261 6301 heartbeater.cc:347] Connected to a master server at 127.10.173.190:46825
I20260811 22:16:46.373642 6301 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:46.374581 6301 heartbeater.cc:511] Master 127.10.173.190:46825 requested a full tablet report, sending...
I20260811 22:16:46.376369 5845 ts_manager.cc:194] Registered new tserver with Master: 9fc3bfbf1f2b44f09dab1fea185a6f07 (127.10.173.130:36873)
I20260811 22:16:46.377302 5845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:52839
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:16:46.426226 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46825
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:46.783905 6309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:46.784277 6309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:46.784487 6309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:46.792976 6309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:46.793242 6309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:46.793447 6309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:16:46.804706 6309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:46825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:46.806182 6309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:46.807688 6309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:16:46.817412 6107 heartbeater.cc:503] Master 127.10.173.190:46825 was elected leader, sending a full tablet report...
W20260811 22:16:46.823608 6318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:46.823674 6320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:46.823714 6323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:46.824541 6309 server_base.cc:1034] running on GCE node
I20260811 22:16:46.825255 6309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:46.826301 6309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:46.827558 6309 hybrid_clock.cc:648] HybridClock initialized: now 1786486606827501 us; error 91 us; skew 500 ppm
I20260811 22:16:46.827921 6309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:46 dist-test-slave-68xk krb5kdc[5782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:46.836553 6309 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:16:46.838224 6309 webserver.cc:460] Webserver started at http://127.10.173.131:33111/ using document root <none> and password file <none>
I20260811 22:16:46.839156 6309 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:46.839318 6309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:46.839687 6309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:46.843009 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "e003c4c3189a431997b07e54acb24004"
format_stamp: "Formatted at 2026-08-11 22:16:46 on dist-test-slave-68xk"
server_key: "ff56c783be8e4c984d234f6aa0000208"
server_key_iv: "a55a18b3b334942a7ed4bad1f83da8f5"
server_key_version: "encryptionkey@0"
I20260811 22:16:46.843852 6309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "e003c4c3189a431997b07e54acb24004"
format_stamp: "Formatted at 2026-08-11 22:16:46 on dist-test-slave-68xk"
server_key: "ff56c783be8e4c984d234f6aa0000208"
server_key_iv: "a55a18b3b334942a7ed4bad1f83da8f5"
server_key_version: "encryptionkey@0"
I20260811 22:16:46.849702 6309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:16:46.853489 6330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:46.855028 6309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:16:46.855237 6309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "e003c4c3189a431997b07e54acb24004"
format_stamp: "Formatted at 2026-08-11 22:16:46 on dist-test-slave-68xk"
server_key: "ff56c783be8e4c984d234f6aa0000208"
server_key_iv: "a55a18b3b334942a7ed4bad1f83da8f5"
server_key_version: "encryptionkey@0"
I20260811 22:16:46.855470 6309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:46.872733 6309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:46.877103 6309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:46.877413 6309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:46.878490 6309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:46.880119 6309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:46.880230 6309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:46.880340 6309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:46.880390 6309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:46.916281 6309 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:33823
I20260811 22:16:46.916324 6483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:33823 every 8 connection(s)
I20260811 22:16:46.917603 6309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:16:46.921052 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 6309
I20260811 22:16:46.921275 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:16:46.921878 10934 external_mini_cluster.cc:1428] Setting key d57ceda994a466b2670965408a2a2822
Aug 11 22:16:46 dist-test-slave-68xk krb5kdc[5782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486606, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:46.939550 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.920304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:37753 (local address 127.10.173.190:46825)
0811 22:16:46.920785 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:46.920790 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:46.921929 (+ 1139us) server_negotiation.cc:407] Connection header received
0811 22:16:46.924236 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:46.924243 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:46.924338 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:46.924470 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:46.926063 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:46.926930 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:46.928422 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:46.928810 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:46.928913 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:46.932761 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:46.932766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:46.932770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:46.932804 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:46.935645 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:46.936791 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:46.936798 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:46.936802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:46.936968 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:46.937535 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:46.937539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:46.937541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:46.937832 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:46.938036 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:46.938857 (+ 821us) server_negotiation.cc:299] Negotiation successful
0811 22:16:46.939095 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":266,"threads_started":1}
I20260811 22:16:46.940618 6489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.920888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:46825 (local address 127.10.173.131:37753)
0811 22:16:46.921748 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:46.921794 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:46.923848 (+ 2054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:46.924665 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:46.924672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:46.925354 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:46.925906 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:46.925923 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:46.927063 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:46.927067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:46.928288 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:46.928300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:46.929065 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:46.929071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:46.929345 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:46.930048 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:16:46.930073 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:46.932551 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:46.935828 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:46.935836 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:46.935849 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:46.936548 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:46.937098 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:46.937104 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:46.937107 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:46.937403 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:46.937980 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:46.937989 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:46.938357 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 22:16:46.938722 (+ 365us) client_negotiation.cc:240] Negotiation successful
0811 22:16:46.939299 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":368,"threads_started":1}
I20260811 22:16:46.942235 6485 heartbeater.cc:347] Connected to a master server at 127.10.173.190:46825
I20260811 22:16:46.942597 6485 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:46.943321 6485 heartbeater.cc:511] Master 127.10.173.190:46825 requested a full tablet report, sending...
I20260811 22:16:46.944806 5845 ts_manager.cc:194] Registered new tserver with Master: e003c4c3189a431997b07e54acb24004 (127.10.173.131:33823)
I20260811 22:16:46.945782 5845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:37753
I20260811 22:16:46.950634 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:16:46.970041 6488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.955564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.10.173.190:46825)
0811 22:16:46.955763 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:46.955772 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:46.956232 (+ 460us) server_negotiation.cc:407] Connection header received
0811 22:16:46.956439 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:46.956444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:46.956520 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:46.956622 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:46.958107 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:46.958903 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:46.960556 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:46.960982 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:46.961095 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:46.963776 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:46.963781 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:46.963785 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:46.963818 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:46.966700 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:46.967519 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:46.967524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:46.967527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:46.967664 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:46.968177 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:46.968191 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:46.968195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:46.968613 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:46.968806 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:46.969332 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 22:16:46.969609 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:16:46.974622 5845 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:16:46.982487 5845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:16:47.028636 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.996387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.10.173.130:36873)
0811 22:16:47.018921 (+ 22534us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:47.018934 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:47.019387 (+ 453us) server_negotiation.cc:407] Connection header received
0811 22:16:47.019516 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:47.019525 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:47.019800 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:47.020003 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:47.020992 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.022690 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.025859 (+ 3169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.027327 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.027505 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.027704 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:47.028069 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 22:16:47.028253 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22303,"thread_start_us":22560,"threads_started":1}
I20260811 22:16:47.028683 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.995760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:36873 (local address 127.0.0.1:42678)
0811 22:16:47.019043 (+ 23283us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:47.019064 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:47.019233 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:47.020348 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:47.020357 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:47.020395 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:47.020769 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:47.020782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.022863 (+ 2081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.022868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:47.025687 (+ 2819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:47.025702 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.027540 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.027547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:47.027768 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.027829 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 22:16:47.027985 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 22:16:47.028190 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23155,"thread_start_us":23624,"threads_started":1}
I20260811 22:16:47.029250 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.996788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38190 (local address 127.10.173.131:33823)
0811 22:16:47.018572 (+ 21784us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:47.018585 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:47.020271 (+ 1686us) server_negotiation.cc:407] Connection header received
0811 22:16:47.020384 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:47.020399 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:47.020571 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:47.020823 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:47.021597 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.023198 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.026021 (+ 2823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.027362 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.027565 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.027733 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:47.028538 (+ 805us) server_negotiation.cc:299] Negotiation successful
0811 22:16:47.028882 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21546,"thread_start_us":21357,"threads_started":1}
I20260811 22:16:47.029306 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.995730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43376 (local address 127.10.173.129:36429)
0811 22:16:47.020540 (+ 24810us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:47.020548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:47.020577 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:16:47.020733 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:47.020742 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:47.020983 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:47.021176 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:47.021881 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.023368 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.025530 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.027635 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.027823 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.027928 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:47.028785 (+ 857us) server_negotiation.cc:299] Negotiation successful
0811 22:16:47.028931 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24621,"thread_start_us":24399,"threads_started":1}
I20260811 22:16:47.029695 6510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.996092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:33823 (local address 127.0.0.1:38190)
0811 22:16:47.019726 (+ 23634us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:47.019745 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:47.019874 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:47.020815 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:47.020820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:47.020855 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:47.021366 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:47.021382 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.023376 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.023381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:47.025885 (+ 2504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:47.025895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.027864 (+ 1969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.027870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:47.028046 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.028075 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:16:47.028167 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:16:47.028266 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23503,"mutex_wait_us":40,"thread_start_us":22695,"threads_started":1}
I20260811 22:16:47.029775 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:46.995549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:36429 (local address 127.0.0.1:43376)
0811 22:16:47.017685 (+ 22136us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:47.017745 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:47.018016 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:47.021247 (+ 3231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:47.021254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:47.021292 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:47.021713 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:47.021722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.023568 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.023575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:47.025358 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:47.025371 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.028135 (+ 2764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.028142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:47.028303 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.028335 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:16:47.028440 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:16:47.028560 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21731,"mutex_wait_us":22,"thread_start_us":21520,"threads_started":1}
I20260811 22:16:47.032621 6208 tablet_service.cc:1467] Processing CreateTablet for tablet 3d1c9d84c4dd433498df1f68242eba38 (DEFAULT_TABLE table=test-table [id=435c029d06434953a46eb96f084051a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:47.033314 6396 tablet_service.cc:1467] Processing CreateTablet for tablet 3d1c9d84c4dd433498df1f68242eba38 (DEFAULT_TABLE table=test-table [id=435c029d06434953a46eb96f084051a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:47.033314 6018 tablet_service.cc:1467] Processing CreateTablet for tablet 3d1c9d84c4dd433498df1f68242eba38 (DEFAULT_TABLE table=test-table [id=435c029d06434953a46eb96f084051a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:47.034067 6208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d1c9d84c4dd433498df1f68242eba38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:47.034763 6018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d1c9d84c4dd433498df1f68242eba38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:47.034778 6396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d1c9d84c4dd433498df1f68242eba38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:47.046547 6516 tablet_bootstrap.cc:492] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: Bootstrap starting.
I20260811 22:16:47.049491 6517 tablet_bootstrap.cc:492] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: Bootstrap starting.
I20260811 22:16:47.050194 6516 tablet_bootstrap.cc:654] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:47.051594 6516 log.cc:824] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:47.053354 6517 tablet_bootstrap.cc:654] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:47.054880 6517 log.cc:824] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:47.057744 6516 tablet_bootstrap.cc:492] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: No bootstrap required, opened a new log
I20260811 22:16:47.055429 6518 tablet_bootstrap.cc:492] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: Bootstrap starting.
I20260811 22:16:47.058099 6516 ts_tablet_manager.cc:1397] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.004s
I20260811 22:16:47.060808 6517 tablet_bootstrap.cc:492] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: No bootstrap required, opened a new log
I20260811 22:16:47.061128 6517 ts_tablet_manager.cc:1397] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20260811 22:16:47.061635 6518 tablet_bootstrap.cc:654] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:47.062812 6518 log.cc:824] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:47.065392 6518 tablet_bootstrap.cc:492] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: No bootstrap required, opened a new log
I20260811 22:16:47.065665 6518 ts_tablet_manager.cc:1397] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 22:16:47.068862 6516 raft_consensus.cc:348] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 } } peers { permanent_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36873 } } peers { permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 } }
I20260811 22:16:47.069343 6516 raft_consensus.cc:374] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:47.069448 6516 raft_consensus.cc:729] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc3bfbf1f2b44f09dab1fea185a6f07, State: Initialized, Role: FOLLOWER
I20260811 22:16:47.070106 6516 consensus_queue.cc:260] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 } } peers { permanent_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36873 } } peers { permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 } }
I20260811 22:16:47.071305 6301 heartbeater.cc:503] Master 127.10.173.190:46825 was elected leader, sending a full tablet report...
I20260811 22:16:47.071976 6517 raft_consensus.cc:348] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 } } peers { permanent_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36873 } } peers { permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 } }
I20260811 22:16:47.072404 6517 raft_consensus.cc:374] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:47.072506 6517 raft_consensus.cc:729] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e003c4c3189a431997b07e54acb24004, State: Initialized, Role: FOLLOWER
I20260811 22:16:47.073273 6517 consensus_queue.cc:260] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 } } peers { permanent_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36873 } } peers { permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 } }
I20260811 22:16:47.073817 6516 ts_tablet_manager.cc:1428] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260811 22:16:47.074024 6518 raft_consensus.cc:348] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 } } peers { permanent_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36873 } } peers { permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 } }
I20260811 22:16:47.074501 6518 raft_consensus.cc:374] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:47.074608 6518 raft_consensus.cc:729] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96c0db61e19e4a4ca5b8331e3e5f8920, State: Initialized, Role: FOLLOWER
I20260811 22:16:47.074980 6485 heartbeater.cc:503] Master 127.10.173.190:46825 was elected leader, sending a full tablet report...
I20260811 22:16:47.075295 6518 consensus_queue.cc:260] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 } } peers { permanent_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36873 } } peers { permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 } }
I20260811 22:16:47.075488 6517 ts_tablet_manager.cc:1428] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 22:16:47.077584 6518 ts_tablet_manager.cc:1428] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
W20260811 22:16:47.095150 6303 tablet.cc:2369] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:16:47.169292 6486 tablet.cc:2369] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:16:47.288695 6108 tablet.cc:2369] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:16:47.430596 6527 raft_consensus.cc:482] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:16:47.430891 6527 raft_consensus.cc:504] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 } } peers { permanent_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36873 } } peers { permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 } }
I20260811 22:16:47.432708 6527 leader_election.cc:288] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96c0db61e19e4a4ca5b8331e3e5f8920 (127.10.173.129:36429), e003c4c3189a431997b07e54acb24004 (127.10.173.131:33823)
I20260811 22:16:47.439524 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:47.433110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:57639 (local address 127.10.173.129:36429)
0811 22:16:47.433345 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:47.433353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:47.434195 (+ 842us) server_negotiation.cc:407] Connection header received
0811 22:16:47.434454 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:47.434462 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:47.434541 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:47.434638 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:47.435179 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.435913 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.437735 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.438598 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.438770 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.438850 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:47.439129 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 22:16:47.439256 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:16:47.439749 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:47.433125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:36429 (local address 127.10.173.130:57639)
0811 22:16:47.434076 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:47.434092 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:47.434281 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:47.434657 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:47.434661 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:47.434685 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:47.435028 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:47.435037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.436052 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.436058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:47.437576 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:47.437589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.438804 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.438810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:47.438958 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.438985 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:16:47.439102 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:16:47.439226 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":817,"mutex_wait_us":7,"thread_start_us":581,"threads_started":1}
I20260811 22:16:47.439842 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:47.433250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:43601 (local address 127.10.173.131:33823)
0811 22:16:47.433479 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:47.433487 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:47.434352 (+ 865us) server_negotiation.cc:407] Connection header received
0811 22:16:47.434567 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:47.434576 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:47.434649 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:47.434753 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:47.435409 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.436105 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.437908 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.438834 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.439018 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.439102 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:47.439443 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 22:16:47.439567 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:16:47.440165 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:47.433247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:33823 (local address 127.10.173.130:43601)
0811 22:16:47.434237 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:47.434252 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:47.434408 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:47.434775 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:47.434780 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:47.434821 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:47.435219 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:47.435228 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.436260 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.436267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:47.437748 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:47.437760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.439085 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.439092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:47.439280 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.439303 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:16:47.439417 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:16:47.439550 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":1,"thread_start_us":628,"threads_started":1}
I20260811 22:16:47.440816 6044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1c9d84c4dd433498df1f68242eba38" candidate_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" is_pre_election: true
I20260811 22:16:47.441154 6423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1c9d84c4dd433498df1f68242eba38" candidate_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e003c4c3189a431997b07e54acb24004" is_pre_election: true
I20260811 22:16:47.441658 6423 raft_consensus.cc:2454] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fc3bfbf1f2b44f09dab1fea185a6f07 in term 0.
I20260811 22:16:47.441663 6044 raft_consensus.cc:2454] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fc3bfbf1f2b44f09dab1fea185a6f07 in term 0.
I20260811 22:16:47.442471 6138 leader_election.cc:302] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fc3bfbf1f2b44f09dab1fea185a6f07, e003c4c3189a431997b07e54acb24004; no voters:
I20260811 22:16:47.442939 6527 raft_consensus.cc:2781] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:16:47.443154 6527 raft_consensus.cc:482] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:16:47.443292 6527 raft_consensus.cc:3037] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:47.445935 6527 raft_consensus.cc:504] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 } } peers { permanent_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36873 } } peers { permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 } }
I20260811 22:16:47.446743 6527 leader_election.cc:288] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [CANDIDATE]: Term 1 election: Requested vote from peers 96c0db61e19e4a4ca5b8331e3e5f8920 (127.10.173.129:36429), e003c4c3189a431997b07e54acb24004 (127.10.173.131:33823)
I20260811 22:16:47.447263 6044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1c9d84c4dd433498df1f68242eba38" candidate_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920"
I20260811 22:16:47.447391 6423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d1c9d84c4dd433498df1f68242eba38" candidate_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e003c4c3189a431997b07e54acb24004"
I20260811 22:16:47.447510 6044 raft_consensus.cc:3037] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:47.447594 6423 raft_consensus.cc:3037] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:47.449719 6423 raft_consensus.cc:2454] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fc3bfbf1f2b44f09dab1fea185a6f07 in term 1.
I20260811 22:16:47.450038 6044 raft_consensus.cc:2454] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fc3bfbf1f2b44f09dab1fea185a6f07 in term 1.
I20260811 22:16:47.450166 6138 leader_election.cc:302] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fc3bfbf1f2b44f09dab1fea185a6f07, e003c4c3189a431997b07e54acb24004; no voters:
I20260811 22:16:47.450508 6527 raft_consensus.cc:2781] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:16:47.450999 6527 raft_consensus.cc:686] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 1 LEADER]: Becoming Leader. State: Replica: 9fc3bfbf1f2b44f09dab1fea185a6f07, State: Running, Role: LEADER
I20260811 22:16:47.451512 6527 consensus_queue.cc:237] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 } } peers { permanent_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36873 } } peers { permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 } }
I20260811 22:16:47.457058 5843 catalog_manager.cc:5495] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fc3bfbf1f2b44f09dab1fea185a6f07 (127.10.173.130). New cstate: current_term: 1 leader_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fc3bfbf1f2b44f09dab1fea185a6f07" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 36873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 } health_report { overall_health: UNKNOWN } } }
I20260811 22:16:47.490240 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:47.484156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42692 (local address 127.10.173.130:36873)
0811 22:16:47.484432 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:47.484439 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:47.484739 (+ 300us) server_negotiation.cc:407] Connection header received
0811 22:16:47.484932 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:47.484936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:47.485064 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:47.485209 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:16:47.485674 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.486473 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.487917 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.488329 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.488426 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.489073 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:16:47.489630 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:16:47.489750 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:47.489857 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 22:16:47.489925 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:16:47.498878 6423 raft_consensus.cc:1261] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004 [term 1 FOLLOWER]: Refusing update from remote peer 9fc3bfbf1f2b44f09dab1fea185a6f07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:16:47.498881 6044 raft_consensus.cc:1261] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920 [term 1 FOLLOWER]: Refusing update from remote peer 9fc3bfbf1f2b44f09dab1fea185a6f07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:16:47.499903 6535 consensus_queue.cc:1035] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96c0db61e19e4a4ca5b8331e3e5f8920" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:16:47.500313 6527 consensus_queue.cc:1035] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e003c4c3189a431997b07e54acb24004" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:16:47.512609 6541 mvcc.cc:205] Tried to move back new op lower bound from 7317449144304730112 to 7317449144126238720. Current Snapshot: MvccSnapshot[applied={T|T < 7317449144304730112}]
I20260811 22:16:47.528239 5844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:16:47.528568 5844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:16:47.532543 5844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0b6d1cca81a419cace997e49ee0e4ef: Sending DeleteTablet for 3 replicas of tablet 3d1c9d84c4dd433498df1f68242eba38
I20260811 22:16:47.533692 6018 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d1c9d84c4dd433498df1f68242eba38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:47 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:43376
I20260811 22:16:47.533785 6208 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d1c9d84c4dd433498df1f68242eba38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:47 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:42678
I20260811 22:16:47.533986 6396 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d1c9d84c4dd433498df1f68242eba38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:47 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:38190
I20260811 22:16:47.541996 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 5930
I20260811 22:16:47.546980 6555 tablet_replica.cc:330] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: stopping tablet replica
I20260811 22:16:47.547366 6554 tablet_replica.cc:330] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: stopping tablet replica
I20260811 22:16:47.548156 6554 raft_consensus.cc:2229] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:16:47.548799 6555 raft_consensus.cc:2229] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:16:47.549731 6555 raft_consensus.cc:2258] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:47.550489 6554 raft_consensus.cc:2258] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:47.553774 6554 ts_tablet_manager.cc:1905] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:47.556591 6555 ts_tablet_manager.cc:1905] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:47.557334 6556 tablet_replica.cc:330] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: stopping tablet replica
I20260811 22:16:47.557850 6556 raft_consensus.cc:2229] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 11 22:16:47 dist-test-slave-68xk krb5kdc[5782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:16:47.559305 6556 raft_consensus.cc:2258] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:47.562000 6556 ts_tablet_manager.cc:1905] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:47.564721 6554 ts_tablet_manager.cc:1918] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:47.564893 6554 log.cc:1196] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/3d1c9d84c4dd433498df1f68242eba38
I20260811 22:16:47.565496 6554 ts_tablet_manager.cc:1939] T 3d1c9d84c4dd433498df1f68242eba38 P 96c0db61e19e4a4ca5b8331e3e5f8920: Deleting consensus metadata
I20260811 22:16:47.567535 5823 catalog_manager.cc:4841] TS 96c0db61e19e4a4ca5b8331e3e5f8920 (127.10.173.129:36429): tablet 3d1c9d84c4dd433498df1f68242eba38 (table test-table [id=435c029d06434953a46eb96f084051a3]) successfully deleted
I20260811 22:16:47.568073 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:47.543684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43390 (local address 127.10.173.129:36429)
0811 22:16:47.543870 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:47.543877 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:47.546985 (+ 3108us) server_negotiation.cc:407] Connection header received
0811 22:16:47.547291 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:47.547297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:47.547389 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:47.547478 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:47.549454 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.550754 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.552766 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.553451 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.553554 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.559594 (+ 6040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:47.559610 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:47.559630 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:47.559677 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:47.563996 (+ 4319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:47.565074 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:47.565084 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:47.565087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:47.565309 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:47.565835 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:47.565840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:47.565844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:47.566333 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:47.566675 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:47.567348 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 22:16:47.567605 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:16:47.569558 6101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:47.574792 6555 ts_tablet_manager.cc:1918] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:47.575042 6555 log.cc:1196] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/3d1c9d84c4dd433498df1f68242eba38
I20260811 22:16:47.575659 6555 ts_tablet_manager.cc:1939] T 3d1c9d84c4dd433498df1f68242eba38 P 9fc3bfbf1f2b44f09dab1fea185a6f07: Deleting consensus metadata
I20260811 22:16:47.575831 6556 ts_tablet_manager.cc:1918] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:47.576021 6556 log.cc:1196] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/3d1c9d84c4dd433498df1f68242eba38
I20260811 22:16:47.576538 6556 ts_tablet_manager.cc:1939] T 3d1c9d84c4dd433498df1f68242eba38 P e003c4c3189a431997b07e54acb24004: Deleting consensus metadata
I20260811 22:16:47.577960 5824 catalog_manager.cc:4841] TS 9fc3bfbf1f2b44f09dab1fea185a6f07 (127.10.173.130:36873): tablet 3d1c9d84c4dd433498df1f68242eba38 (table test-table [id=435c029d06434953a46eb96f084051a3]) successfully deleted
I20260811 22:16:47.578274 5820 catalog_manager.cc:4841] TS e003c4c3189a431997b07e54acb24004 (127.10.173.131:33823): tablet 3d1c9d84c4dd433498df1f68242eba38 (table test-table [id=435c029d06434953a46eb96f084051a3]) successfully deleted
I20260811 22:16:47.710487 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 5930
I20260811 22:16:47.732748 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 6113
Aug 11 22:16:47 dist-test-slave-68xk krb5kdc[5782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:16:47.752604 6514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:47.733621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.10.173.130:36873)
0811 22:16:47.733828 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:47.733834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:47.733858 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:16:47.733999 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:47.734005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:47.734116 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:47.734255 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:47.735609 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.736948 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.738608 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.739335 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.739517 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.744890 (+ 5373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:47.744915 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:47.744949 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:47.744994 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:47.749429 (+ 4435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:47.750194 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:47.750204 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:47.750208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:47.750392 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:47.750837 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:47.750842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:47.750845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:47.751355 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:47.751713 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:47.751971 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 22:16:47.752196 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 22:16:47.753379 6295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:47.873093 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 6113
I20260811 22:16:47.902756 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 6309
Aug 11 22:16:47 dist-test-slave-68xk krb5kdc[5782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:16:47.919931 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:47.903632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38194 (local address 127.10.173.131:33823)
0811 22:16:47.903848 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:47.903853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:47.903875 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:16:47.904078 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:47.904082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:47.904156 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:47.904262 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:47.905774 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.906681 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.908058 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:47.908498 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:47.908604 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:47.913481 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:47.913499 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:47.913516 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:47.913560 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:47.916617 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:47.917607 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:47.917618 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:47.917621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:47.917778 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:47.918273 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:47.918277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:47.918279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:47.918761 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:47.918981 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:47.919372 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 22:16:47.919575 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:16:47.920648 6477 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:48.011668 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 6309
I20260811 22:16:48.034315 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 5797
I20260811 22:16:48.035725 5883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:48.145694 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 5797
2026-08-11T22:16:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5305 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T22:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:16:48Z Disabled control of system clock
I20260811 22:16:48.243999 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:44473
--webserver_interface=127.10.173.190
--webserver_port=0
--builtin_ntp_servers=127.10.173.148:38927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:44473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:16:48.608718 6568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:48.609050 6568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:48.609145 6568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:48.616755 6568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:16:48.616871 6568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:48.616910 6568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:48.616976 6568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:16:48.617022 6568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:16:48.626582 6568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:44473
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:44473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:48.627897 6568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:48.629680 6568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:48.642113 6576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:48.642299 6578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:48.642880 6575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:48.643138 6568 server_base.cc:1034] running on GCE node
I20260811 22:16:48.644073 6568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:48.645536 6568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:48.646735 6568 hybrid_clock.cc:648] HybridClock initialized: now 1786486608646698 us; error 39 us; skew 500 ppm
I20260811 22:16:48.647053 6568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:16:48.649302 6568 webserver.cc:460] Webserver started at http://127.10.173.190:40441/ using document root <none> and password file <none>
I20260811 22:16:48.649950 6568 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:48.650039 6568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:48.650315 6568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:48.653280 6568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "906ba0b77b914addba37fafdbb07bc1d"
format_stamp: "Formatted at 2026-08-11 22:16:48 on dist-test-slave-68xk"
server_key: "f40c463186577490aee9dcc016ce4f3d"
server_key_iv: "087d3e975877c0b2aebb7b3fc933ed0a"
server_key_version: "encryptionkey@0"
I20260811 22:16:48.654039 6568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "906ba0b77b914addba37fafdbb07bc1d"
format_stamp: "Formatted at 2026-08-11 22:16:48 on dist-test-slave-68xk"
server_key: "f40c463186577490aee9dcc016ce4f3d"
server_key_iv: "087d3e975877c0b2aebb7b3fc933ed0a"
server_key_version: "encryptionkey@0"
I20260811 22:16:48.659353 6568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 22:16:48.663710 6587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:48.665277 6568 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260811 22:16:48.665478 6568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "906ba0b77b914addba37fafdbb07bc1d"
format_stamp: "Formatted at 2026-08-11 22:16:48 on dist-test-slave-68xk"
server_key: "f40c463186577490aee9dcc016ce4f3d"
server_key_iv: "087d3e975877c0b2aebb7b3fc933ed0a"
server_key_version: "encryptionkey@0"
I20260811 22:16:48.665704 6568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:48.691033 6568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:48.691982 6568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:48.692313 6568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:48.712179 6568 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:44473
I20260811 22:16:48.712270 6659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:44473 every 8 connection(s)
I20260811 22:16:48.713539 6568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:16:48.715692 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 6568
I20260811 22:16:48.715890 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:16:48.716356 10934 external_mini_cluster.cc:1428] Setting key de266c1bac7d5eba84c3f6ea3ce46517
I20260811 22:16:48.718245 6661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:48.730909 6665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:48.718225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37556 (local address 127.10.173.190:44473)
0811 22:16:48.719334 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:48.719353 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:48.719396 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:16:48.721054 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:48.721083 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:48.721534 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:48.721866 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:48.722629 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:48.723997 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:48.725991 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:48.726786 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:48.726942 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:48.728355 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:48.728363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:48.728380 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:48.728965 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:48.729057 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:48.729084 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:48.729191 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:48.729441 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 22:16:48.729813 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":458,"threads_started":1}
I20260811 22:16:48.731945 6661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d: Bootstrap starting.
I20260811 22:16:48.735180 6661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:48.736209 6661 log.cc:824] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:48.739078 6661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d: No bootstrap required, opened a new log
I20260811 22:16:48.747067 6661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "906ba0b77b914addba37fafdbb07bc1d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44473 } }
I20260811 22:16:48.747467 6661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:48.747594 6661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906ba0b77b914addba37fafdbb07bc1d, State: Initialized, Role: FOLLOWER
I20260811 22:16:48.748163 6661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "906ba0b77b914addba37fafdbb07bc1d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44473 } }
I20260811 22:16:48.748401 6661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:16:48.748561 6661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:16:48.748764 6661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:48.751363 6661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "906ba0b77b914addba37fafdbb07bc1d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44473 } }
I20260811 22:16:48.751902 6661 leader_election.cc:302] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 906ba0b77b914addba37fafdbb07bc1d; no voters:
I20260811 22:16:48.752516 6661 leader_election.cc:288] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:16:48.752702 6669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:16:48.753430 6669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [term 1 LEADER]: Becoming Leader. State: Replica: 906ba0b77b914addba37fafdbb07bc1d, State: Running, Role: LEADER
I20260811 22:16:48.754026 6669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "906ba0b77b914addba37fafdbb07bc1d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44473 } }
I20260811 22:16:48.754667 6661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:16:48.756886 6671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 906ba0b77b914addba37fafdbb07bc1d. Latest consensus state: current_term: 1 leader_uuid: "906ba0b77b914addba37fafdbb07bc1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "906ba0b77b914addba37fafdbb07bc1d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44473 } } }
I20260811 22:16:48.757649 6671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:48.757396 6670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "906ba0b77b914addba37fafdbb07bc1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "906ba0b77b914addba37fafdbb07bc1d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44473 } } }
I20260811 22:16:48.762337 6670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:48.763814 6679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:16:48.770195 6679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:16:48.778550 6679 catalog_manager.cc:1347] Generated new cluster ID: 93b3f0f4825f4042befc06bcacac1510
I20260811 22:16:48.778698 6679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:16:48.788856 6679 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:16:48.790634 6679 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:16:48.805187 6679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d: Generated new TSK 0
I20260811 22:16:48.806061 6679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:16:48.816274 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44473
--builtin_ntp_servers=127.10.173.148:38927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:16:49.190164 6692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:49.190611 6692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:49.190726 6692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:49.198737 6692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:49.198825 6692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:49.198930 6692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:16:49.207906 6692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:49.209128 6692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:49.211001 6692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:49.223978 6701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:49.223966 6707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:49.224205 6702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:49.224815 6692 server_base.cc:1034] running on GCE node
I20260811 22:16:49.225421 6692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:49.226351 6692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:49.227684 6692 hybrid_clock.cc:648] HybridClock initialized: now 1786486609227640 us; error 33 us; skew 500 ppm
I20260811 22:16:49.228082 6692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:16:49.231204 6692 webserver.cc:460] Webserver started at http://127.10.173.129:36397/ using document root <none> and password file <none>
I20260811 22:16:49.232300 6692 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:49.232463 6692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:49.232899 6692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:49.237067 6692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "245cb2753dd9444bac14b1f9cbd4201c"
format_stamp: "Formatted at 2026-08-11 22:16:49 on dist-test-slave-68xk"
server_key: "a52c60fce4f5a8b24934a2d5b9453d89"
server_key_iv: "7405244389e117567604851c75f0635c"
server_key_version: "encryptionkey@0"
I20260811 22:16:49.238076 6692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "245cb2753dd9444bac14b1f9cbd4201c"
format_stamp: "Formatted at 2026-08-11 22:16:49 on dist-test-slave-68xk"
server_key: "a52c60fce4f5a8b24934a2d5b9453d89"
server_key_iv: "7405244389e117567604851c75f0635c"
server_key_version: "encryptionkey@0"
I20260811 22:16:49.244020 6692 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 22:16:49.248047 6713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:49.249449 6692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:16:49.249675 6692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "245cb2753dd9444bac14b1f9cbd4201c"
format_stamp: "Formatted at 2026-08-11 22:16:49 on dist-test-slave-68xk"
server_key: "a52c60fce4f5a8b24934a2d5b9453d89"
server_key_iv: "7405244389e117567604851c75f0635c"
server_key_version: "encryptionkey@0"
I20260811 22:16:49.249924 6692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:49.272420 6692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:49.273382 6692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:49.273746 6692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:49.274910 6692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:49.276626 6692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:49.276768 6692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:49.276924 6692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:49.277029 6692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:49.315114 6692 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:40949
I20260811 22:16:49.315217 6869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:40949 every 8 connection(s)
I20260811 22:16:49.316553 6692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:16:49.322935 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 6692
I20260811 22:16:49.323156 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:16:49.323794 10934 external_mini_cluster.cc:1428] Setting key 8f064ad6cedf8298631e88ff936f17a3
I20260811 22:16:49.327342 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44473
--builtin_ntp_servers=127.10.173.148:38927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:16:49.332283 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:49.319586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:49885 (local address 127.10.173.190:44473)
0811 22:16:49.320119 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:49.320124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:49.321194 (+ 1070us) server_negotiation.cc:407] Connection header received
0811 22:16:49.323428 (+ 2234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:49.323435 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:49.323543 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:49.323668 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:49.325016 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:49.326108 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:49.328918 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:49.329384 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:49.329501 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:49.331382 (+ 1881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:49.331388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:49.331392 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:49.331414 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:49.331459 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:49.331473 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:49.331553 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:49.331912 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 22:16:49.332013 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":278,"threads_started":1}
I20260811 22:16:49.333914 6873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:49.320021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:44473 (local address 127.10.173.129:49885)
0811 22:16:49.320983 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:49.321028 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:49.323001 (+ 1973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:49.323946 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:49.323957 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:49.324117 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:49.324816 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:49.324846 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:49.326287 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:49.326293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:49.328725 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:49.328747 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:49.329659 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:49.329670 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:49.330038 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:49.330952 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:16:49.330958 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:49.331020 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:16:49.331148 (+ 128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:49.331593 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:49.331608 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:49.331620 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:16:49.331844 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 22:16:49.332328 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":293,"threads_started":1}
I20260811 22:16:49.335784 6870 heartbeater.cc:347] Connected to a master server at 127.10.173.190:44473
I20260811 22:16:49.336164 6870 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:49.337206 6870 heartbeater.cc:511] Master 127.10.173.190:44473 requested a full tablet report, sending...
I20260811 22:16:49.339584 6611 ts_manager.cc:194] Registered new tserver with Master: 245cb2753dd9444bac14b1f9cbd4201c (127.10.173.129:40949)
I20260811 22:16:49.341274 6611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.129:49885
W20260811 22:16:49.676283 6875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:49.676570 6875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:49.676658 6875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:49.684087 6875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:49.684191 6875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:49.684300 6875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:16:49.693493 6875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:49.694777 6875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:49.696506 6875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:49.710127 6883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:49.710189 6890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:49.710230 6886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:49.710968 6875 server_base.cc:1034] running on GCE node
I20260811 22:16:49.711556 6875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:49.712561 6875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:49.713807 6875 hybrid_clock.cc:648] HybridClock initialized: now 1786486609713745 us; error 50 us; skew 500 ppm
I20260811 22:16:49.714139 6875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:16:49.716495 6875 webserver.cc:460] Webserver started at http://127.10.173.130:39169/ using document root <none> and password file <none>
I20260811 22:16:49.717206 6875 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:49.717342 6875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:49.717664 6875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:49.720563 6875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "703d7428a2434cdf892d6e4dc0a7040a"
format_stamp: "Formatted at 2026-08-11 22:16:49 on dist-test-slave-68xk"
server_key: "7f61b783681c557237c13e19d7295710"
server_key_iv: "e0073398995699eba58e8dce5f29bf60"
server_key_version: "encryptionkey@0"
I20260811 22:16:49.721381 6875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "703d7428a2434cdf892d6e4dc0a7040a"
format_stamp: "Formatted at 2026-08-11 22:16:49 on dist-test-slave-68xk"
server_key: "7f61b783681c557237c13e19d7295710"
server_key_iv: "e0073398995699eba58e8dce5f29bf60"
server_key_version: "encryptionkey@0"
I20260811 22:16:49.726722 6875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:16:49.730324 6896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:49.731772 6875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:16:49.731966 6875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "703d7428a2434cdf892d6e4dc0a7040a"
format_stamp: "Formatted at 2026-08-11 22:16:49 on dist-test-slave-68xk"
server_key: "7f61b783681c557237c13e19d7295710"
server_key_iv: "e0073398995699eba58e8dce5f29bf60"
server_key_version: "encryptionkey@0"
I20260811 22:16:49.732204 6875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:49.754592 6875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:49.755554 6875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:49.755887 6875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:49.756937 6875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:49.758633 6875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:49.758766 6875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:49.758891 6875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:49.758978 6875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:49.796190 6875 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:34465
I20260811 22:16:49.796310 7045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:34465 every 8 connection(s)
I20260811 22:16:49.797950 6875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:16:49.802000 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 6875
I20260811 22:16:49.802251 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:16:49.802944 10934 external_mini_cluster.cc:1428] Setting key 554b9da942367f581deb1433fd037d3a
I20260811 22:16:49.806432 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44473
--builtin_ntp_servers=127.10.173.148:38927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:16:49.813822 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:49.800990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:54721 (local address 127.10.173.190:44473)
0811 22:16:49.801217 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:49.801222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:49.802887 (+ 1665us) server_negotiation.cc:407] Connection header received
0811 22:16:49.805352 (+ 2465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:49.805360 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:49.805453 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:49.805666 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:49.807227 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:49.808017 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:49.810773 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:49.811198 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:49.811316 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:49.813056 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:49.813061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:49.813065 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:49.813082 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:49.813111 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:49.813122 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:49.813194 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:49.813505 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 22:16:49.813582 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 22:16:49.815447 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:49.801352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:44473 (local address 127.10.173.130:54721)
0811 22:16:49.802665 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:49.802726 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:49.804992 (+ 2266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:49.805919 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:49.805932 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:49.806064 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:49.807011 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:49.807041 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:49.808201 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:49.808211 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:49.810594 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:49.810614 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:49.811449 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:49.811458 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:49.811819 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:49.812673 (+ 854us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:16:49.812680 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:49.812737 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:16:49.812855 (+ 118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:49.813227 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:49.813240 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:49.813252 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:16:49.813474 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 22:16:49.813923 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":366,"threads_started":1}
I20260811 22:16:49.816938 7046 heartbeater.cc:347] Connected to a master server at 127.10.173.190:44473
I20260811 22:16:49.817291 7046 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:49.818282 7046 heartbeater.cc:511] Master 127.10.173.190:44473 requested a full tablet report, sending...
I20260811 22:16:49.819995 6611 ts_manager.cc:194] Registered new tserver with Master: 703d7428a2434cdf892d6e4dc0a7040a (127.10.173.130:34465)
I20260811 22:16:49.820839 6611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.130:54721
W20260811 22:16:50.152808 7049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:50.153090 7049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:50.153228 7049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:50.160806 7049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:50.160900 7049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:50.161063 7049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:16:50.170008 7049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:50.171303 7049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:50.173166 7049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:50.186401 7055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:50.186360 7061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:50.186774 7056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:50.191993 7049 server_base.cc:1034] running on GCE node
I20260811 22:16:50.192624 7049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:50.193651 7049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:50.194897 7049 hybrid_clock.cc:648] HybridClock initialized: now 1786486610194850 us; error 63 us; skew 500 ppm
I20260811 22:16:50.195235 7049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:16:50.197643 7049 webserver.cc:460] Webserver started at http://127.10.173.131:46119/ using document root <none> and password file <none>
I20260811 22:16:50.198402 7049 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:50.198571 7049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:50.198900 7049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:50.201778 7049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "945073fe636d43b18a08437c312aae29"
format_stamp: "Formatted at 2026-08-11 22:16:50 on dist-test-slave-68xk"
server_key: "70f9db5dc844a4a102e598300fd23821"
server_key_iv: "256fecaccdfdda035654e48ac767e23a"
server_key_version: "encryptionkey@0"
I20260811 22:16:50.202592 7049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "945073fe636d43b18a08437c312aae29"
format_stamp: "Formatted at 2026-08-11 22:16:50 on dist-test-slave-68xk"
server_key: "70f9db5dc844a4a102e598300fd23821"
server_key_iv: "256fecaccdfdda035654e48ac767e23a"
server_key_version: "encryptionkey@0"
I20260811 22:16:50.207582 7049 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:16:50.211011 7068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:50.212311 7049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:16:50.212487 7049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "945073fe636d43b18a08437c312aae29"
format_stamp: "Formatted at 2026-08-11 22:16:50 on dist-test-slave-68xk"
server_key: "70f9db5dc844a4a102e598300fd23821"
server_key_iv: "256fecaccdfdda035654e48ac767e23a"
server_key_version: "encryptionkey@0"
I20260811 22:16:50.212713 7049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:50.237813 7049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:50.238724 7049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:50.239059 7049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:50.240149 7049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:50.241680 7049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:50.241813 7049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:50.241952 7049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:50.242050 7049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:50.278105 7049 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:43791
I20260811 22:16:50.278203 7228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:43791 every 8 connection(s)
I20260811 22:16:50.279512 7049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:16:50.280653 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 7049
I20260811 22:16:50.280859 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:16:50.281417 10934 external_mini_cluster.cc:1428] Setting key 5ad3f177e26e8e8b28cfb21a25f8120b
I20260811 22:16:50.292402 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.282509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:37551 (local address 127.10.173.190:44473)
0811 22:16:50.282699 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:50.282707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:50.284645 (+ 1938us) server_negotiation.cc:407] Connection header received
0811 22:16:50.286496 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:50.286501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:50.286568 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:50.286650 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:50.287638 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.288402 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.289952 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.290323 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.290480 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.291711 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:50.291716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:50.291720 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:50.291737 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:50.291769 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:50.291780 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:50.291850 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:50.292047 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 22:16:50.292123 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:16:50.293356 7231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.283221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:44473 (local address 127.10.173.131:37551)
0811 22:16:50.284470 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:50.284537 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:50.286213 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:50.286823 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:50.286830 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:50.286923 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:50.287493 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.287508 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.288541 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.288545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.289829 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.289841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.290563 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.290568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.290793 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.291416 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:16:50.291419 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:50.291455 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:16:50.291532 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:50.291864 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:50.291873 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:50.291878 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:16:50.292019 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 22:16:50.292305 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":392,"threads_started":1}
I20260811 22:16:50.294776 7229 heartbeater.cc:347] Connected to a master server at 127.10.173.190:44473
I20260811 22:16:50.295074 7229 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:50.295923 7229 heartbeater.cc:511] Master 127.10.173.190:44473 requested a full tablet report, sending...
I20260811 22:16:50.297546 6611 ts_manager.cc:194] Registered new tserver with Master: 945073fe636d43b18a08437c312aae29 (127.10.173.131:43791)
I20260811 22:16:50.298334 6611 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.131:37551
I20260811 22:16:50.309217 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:16:50.319861 6872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.313408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37572 (local address 127.10.173.190:44473)
0811 22:16:50.313598 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:50.313606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:50.314054 (+ 448us) server_negotiation.cc:407] Connection header received
0811 22:16:50.314274 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:50.314279 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:50.314356 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:50.314480 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:50.315028 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.315721 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.317319 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.317777 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.317918 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.319138 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:50.319142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:50.319146 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:50.319164 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:50.319195 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:50.319206 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:50.319303 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:50.319456 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 22:16:50.319548 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:16:50.323915 6611 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37572:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:16:50.330921 6611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:16:50.344774 6870 heartbeater.cc:503] Master 127.10.173.190:44473 was elected leader, sending a full tablet report...
I20260811 22:16:50.357774 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.343819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41098 (local address 127.10.173.131:43791)
0811 22:16:50.344643 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:50.344666 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:50.345783 (+ 1117us) server_negotiation.cc:407] Connection header received
0811 22:16:50.346113 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:50.346121 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:50.346388 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:50.346600 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:50.347348 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.349549 (+ 2201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.353393 (+ 3844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.355464 (+ 2071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.355720 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.356013 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:50.356977 (+ 964us) server_negotiation.cc:299] Negotiation successful
0811 22:16:50.357246 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":352,"threads_started":1}
I20260811 22:16:50.359077 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.343040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:43791 (local address 127.0.0.1:41098)
0811 22:16:50.345625 (+ 2585us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:50.345663 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:50.345889 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:50.346727 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:50.346736 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:50.346767 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:50.347166 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.347178 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.349766 (+ 2588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.349776 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.353102 (+ 3326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.353125 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.356326 (+ 3201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.356335 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.356640 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.356709 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 22:16:50.358208 (+ 1499us) client_negotiation.cc:240] Negotiation successful
0811 22:16:50.358564 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2373,"mutex_wait_us":4,"thread_start_us":2220,"threads_started":1}
I20260811 22:16:50.361814 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.343420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:34465 (local address 127.0.0.1:53750)
0811 22:16:50.348130 (+ 4710us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:50.348152 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:50.348378 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:50.349713 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:50.349720 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:50.349758 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:50.351726 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.351748 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.354246 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.354254 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.358534 (+ 4280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.358566 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.361033 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.361039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.361238 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.361280 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 22:16:50.361386 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:16:50.361511 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4535,"thread_start_us":4368,"threads_started":1}
I20260811 22:16:50.361991 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.343549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53750 (local address 127.10.173.130:34465)
0811 22:16:50.345094 (+ 1545us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:50.345100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:50.348268 (+ 3168us) server_negotiation.cc:407] Connection header received
0811 22:16:50.348842 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:50.348853 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:50.349139 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:50.349333 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:50.352007 (+ 2674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.354019 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.358834 (+ 4815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.360855 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.361056 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.361288 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:50.361456 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:16:50.361624 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1391,"thread_start_us":1281,"threads_started":1}
I20260811 22:16:50.364358 7137 tablet_service.cc:1467] Processing CreateTablet for tablet 4996740edf234ee492643b15d461fae7 (DEFAULT_TABLE table=test-table [id=e0a28edfce6f435e90657b421c27f1bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:50.365995 7137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4996740edf234ee492643b15d461fae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:50.365962 6960 tablet_service.cc:1467] Processing CreateTablet for tablet 4996740edf234ee492643b15d461fae7 (DEFAULT_TABLE table=test-table [id=e0a28edfce6f435e90657b421c27f1bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:50.368072 6960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4996740edf234ee492643b15d461fae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:50.370617 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.348028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40949 (local address 127.0.0.1:40628)
0811 22:16:50.359579 (+ 11551us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:50.359603 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:50.359725 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:50.363181 (+ 3456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:50.363187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:50.363221 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:50.365210 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.365220 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.366627 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.366634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.368294 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.368308 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.369709 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.369718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.369955 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.369993 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 22:16:50.370081 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:16:50.370226 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11382,"thread_start_us":11937,"threads_started":1}
I20260811 22:16:50.370674 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.348282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40628 (local address 127.10.173.129:40949)
0811 22:16:50.362551 (+ 14269us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:50.362557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:50.362604 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:16:50.362724 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:50.362732 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:50.362961 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:50.363194 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:50.365358 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.366487 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.368452 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.369522 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.369743 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.369967 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:50.370130 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 22:16:50.370301 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14044,"thread_start_us":13906,"threads_started":1}
I20260811 22:16:50.373759 6779 tablet_service.cc:1467] Processing CreateTablet for tablet 4996740edf234ee492643b15d461fae7 (DEFAULT_TABLE table=test-table [id=e0a28edfce6f435e90657b421c27f1bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:50.375278 6779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4996740edf234ee492643b15d461fae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:50.381588 7255 tablet_bootstrap.cc:492] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: Bootstrap starting.
I20260811 22:16:50.383637 7258 tablet_bootstrap.cc:492] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: Bootstrap starting.
I20260811 22:16:50.385965 7255 tablet_bootstrap.cc:654] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:50.386873 7258 tablet_bootstrap.cc:654] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:50.387322 7255 log.cc:824] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:50.387921 7260 tablet_bootstrap.cc:492] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c: Bootstrap starting.
I20260811 22:16:50.388136 7258 log.cc:824] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:50.391512 7258 tablet_bootstrap.cc:492] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: No bootstrap required, opened a new log
I20260811 22:16:50.391736 7260 tablet_bootstrap.cc:654] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:50.391827 7258 ts_tablet_manager.cc:1397] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 22:16:50.392230 7255 tablet_bootstrap.cc:492] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: No bootstrap required, opened a new log
I20260811 22:16:50.392504 7255 ts_tablet_manager.cc:1397] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 22:16:50.392984 7260 log.cc:824] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:50.397403 7260 tablet_bootstrap.cc:492] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c: No bootstrap required, opened a new log
I20260811 22:16:50.397701 7260 ts_tablet_manager.cc:1397] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260811 22:16:50.400638 7255 raft_consensus.cc:348] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 } } peers { permanent_uuid: "703d7428a2434cdf892d6e4dc0a7040a" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34465 } } peers { permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 } }
I20260811 22:16:50.401027 7255 raft_consensus.cc:374] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:50.401135 7255 raft_consensus.cc:729] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945073fe636d43b18a08437c312aae29, State: Initialized, Role: FOLLOWER
I20260811 22:16:50.401772 7255 consensus_queue.cc:260] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 } } peers { permanent_uuid: "703d7428a2434cdf892d6e4dc0a7040a" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34465 } } peers { permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 } }
I20260811 22:16:50.402944 7229 heartbeater.cc:503] Master 127.10.173.190:44473 was elected leader, sending a full tablet report...
I20260811 22:16:50.403477 7255 ts_tablet_manager.cc:1428] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 22:16:50.402715 7258 raft_consensus.cc:348] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 } } peers { permanent_uuid: "703d7428a2434cdf892d6e4dc0a7040a" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34465 } } peers { permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 } }
I20260811 22:16:50.404775 7258 raft_consensus.cc:374] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:50.404911 7258 raft_consensus.cc:729] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 703d7428a2434cdf892d6e4dc0a7040a, State: Initialized, Role: FOLLOWER
I20260811 22:16:50.405553 7258 consensus_queue.cc:260] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 } } peers { permanent_uuid: "703d7428a2434cdf892d6e4dc0a7040a" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34465 } } peers { permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 } }
I20260811 22:16:50.406760 7046 heartbeater.cc:503] Master 127.10.173.190:44473 was elected leader, sending a full tablet report...
I20260811 22:16:50.407472 7258 ts_tablet_manager.cc:1428] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260811 22:16:50.409951 7260 raft_consensus.cc:348] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 } } peers { permanent_uuid: "703d7428a2434cdf892d6e4dc0a7040a" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34465 } } peers { permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 } }
I20260811 22:16:50.410449 7260 raft_consensus.cc:374] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:50.410578 7260 raft_consensus.cc:729] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245cb2753dd9444bac14b1f9cbd4201c, State: Initialized, Role: FOLLOWER
I20260811 22:16:50.411199 7260 consensus_queue.cc:260] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 } } peers { permanent_uuid: "703d7428a2434cdf892d6e4dc0a7040a" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34465 } } peers { permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 } }
I20260811 22:16:50.412860 7260 ts_tablet_manager.cc:1428] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
W20260811 22:16:50.531364 7230 tablet.cc:2369] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:16:50.550348 7047 tablet.cc:2369] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:16:50.569458 6871 tablet.cc:2369] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:16:50.648658 7266 raft_consensus.cc:482] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:16:50.648959 7266 raft_consensus.cc:504] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 } } peers { permanent_uuid: "703d7428a2434cdf892d6e4dc0a7040a" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34465 } } peers { permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 } }
I20260811 22:16:50.650959 7266 leader_election.cc:288] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945073fe636d43b18a08437c312aae29 (127.10.173.131:43791), 245cb2753dd9444bac14b1f9cbd4201c (127.10.173.129:40949)
I20260811 22:16:50.659130 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.651578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:52431 (local address 127.10.173.131:43791)
0811 22:16:50.651962 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:50.651970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:50.652409 (+ 439us) server_negotiation.cc:407] Connection header received
0811 22:16:50.652699 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:50.652705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:50.652802 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:50.652878 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:50.653717 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.654726 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.657024 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.657914 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.658159 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.658243 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:50.658663 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 22:16:50.658819 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260811 22:16:50.659191 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.651332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:43791 (local address 127.10.173.130:52431)
0811 22:16:50.652246 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:50.652270 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:50.652454 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:50.653104 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:50.653109 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:50.653145 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:50.653540 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.653552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.654870 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.654876 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.656806 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.656820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.658203 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.658213 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.658446 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.658501 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 22:16:50.658600 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 22:16:50.658763 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":524,"threads_started":1}
I20260811 22:16:50.659588 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.652636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:49675 (local address 127.10.173.129:40949)
0811 22:16:50.652851 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:50.652855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:50.653035 (+ 180us) server_negotiation.cc:407] Connection header received
0811 22:16:50.653329 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:50.653338 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:50.653430 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:50.653518 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:50.654158 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.655283 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.657259 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.658400 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.658629 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.658703 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:50.659153 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 22:16:50.659306 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:16:50.659682 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.652272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40949 (local address 127.10.173.130:49675)
0811 22:16:50.652933 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:50.652952 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:50.653094 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:50.653569 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:50.653574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:50.653604 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:50.653973 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.653981 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.655438 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.655442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.657042 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:50.657058 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.658726 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.658734 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:50.658930 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.658970 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:16:50.659082 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:16:50.659371 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":354,"threads_started":1}
I20260811 22:16:50.660724 7165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4996740edf234ee492643b15d461fae7" candidate_uuid: "703d7428a2434cdf892d6e4dc0a7040a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945073fe636d43b18a08437c312aae29" is_pre_election: true
I20260811 22:16:50.660861 6809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4996740edf234ee492643b15d461fae7" candidate_uuid: "703d7428a2434cdf892d6e4dc0a7040a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245cb2753dd9444bac14b1f9cbd4201c" is_pre_election: true
I20260811 22:16:50.661439 7165 raft_consensus.cc:2454] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 703d7428a2434cdf892d6e4dc0a7040a in term 0.
I20260811 22:16:50.661439 6809 raft_consensus.cc:2454] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 703d7428a2434cdf892d6e4dc0a7040a in term 0.
I20260811 22:16:50.662200 6899 leader_election.cc:302] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245cb2753dd9444bac14b1f9cbd4201c, 703d7428a2434cdf892d6e4dc0a7040a; no voters:
I20260811 22:16:50.662679 7266 raft_consensus.cc:2781] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:16:50.662853 7266 raft_consensus.cc:482] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:16:50.662977 7266 raft_consensus.cc:3037] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:50.665408 7266 raft_consensus.cc:504] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 } } peers { permanent_uuid: "703d7428a2434cdf892d6e4dc0a7040a" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34465 } } peers { permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 } }
I20260811 22:16:50.666065 7266 leader_election.cc:288] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [CANDIDATE]: Term 1 election: Requested vote from peers 945073fe636d43b18a08437c312aae29 (127.10.173.131:43791), 245cb2753dd9444bac14b1f9cbd4201c (127.10.173.129:40949)
I20260811 22:16:50.666647 7165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4996740edf234ee492643b15d461fae7" candidate_uuid: "703d7428a2434cdf892d6e4dc0a7040a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945073fe636d43b18a08437c312aae29"
I20260811 22:16:50.666708 6809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4996740edf234ee492643b15d461fae7" candidate_uuid: "703d7428a2434cdf892d6e4dc0a7040a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245cb2753dd9444bac14b1f9cbd4201c"
I20260811 22:16:50.666860 7165 raft_consensus.cc:3037] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:50.666990 6809 raft_consensus.cc:3037] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:50.669387 6809 raft_consensus.cc:2454] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 703d7428a2434cdf892d6e4dc0a7040a in term 1.
I20260811 22:16:50.669544 7165 raft_consensus.cc:2454] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 703d7428a2434cdf892d6e4dc0a7040a in term 1.
I20260811 22:16:50.669853 6899 leader_election.cc:302] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 245cb2753dd9444bac14b1f9cbd4201c, 703d7428a2434cdf892d6e4dc0a7040a; no voters:
I20260811 22:16:50.670243 7266 raft_consensus.cc:2781] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:16:50.670786 7266 raft_consensus.cc:686] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 1 LEADER]: Becoming Leader. State: Replica: 703d7428a2434cdf892d6e4dc0a7040a, State: Running, Role: LEADER
I20260811 22:16:50.671344 7266 consensus_queue.cc:237] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 } } peers { permanent_uuid: "703d7428a2434cdf892d6e4dc0a7040a" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34465 } } peers { permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 } }
I20260811 22:16:50.676345 6611 catalog_manager.cc:5495] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a reported cstate change: term changed from 0 to 1, leader changed from <none> to 703d7428a2434cdf892d6e4dc0a7040a (127.10.173.130). New cstate: current_term: 1 leader_uuid: "703d7428a2434cdf892d6e4dc0a7040a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "703d7428a2434cdf892d6e4dc0a7040a" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 34465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 } health_report { overall_health: UNKNOWN } } }
I20260811 22:16:50.724612 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.718582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53752 (local address 127.10.173.130:34465)
0811 22:16:50.718816 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:50.718823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:50.718844 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:16:50.719090 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:50.719096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:50.719204 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:50.719381 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:16:50.719797 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.720606 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.722177 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.722712 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.722856 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.723437 (+ 581us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:16:50.723975 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:16:50.724157 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:50.724227 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 22:16:50.724312 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 22:16:50.733765 7165 raft_consensus.cc:1261] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29 [term 1 FOLLOWER]: Refusing update from remote peer 703d7428a2434cdf892d6e4dc0a7040a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:16:50.733765 6809 raft_consensus.cc:1261] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c [term 1 FOLLOWER]: Refusing update from remote peer 703d7428a2434cdf892d6e4dc0a7040a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:16:50.734912 7270 consensus_queue.cc:1035] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [LEADER]: Connected to new peer: Peer: permanent_uuid: "245cb2753dd9444bac14b1f9cbd4201c" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 40949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:16:50.735369 7266 consensus_queue.cc:1035] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [LEADER]: Connected to new peer: Peer: permanent_uuid: "945073fe636d43b18a08437c312aae29" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 43791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:16:50.747467 7273 mvcc.cc:205] Tried to move back new op lower bound from 7317449157553045504 to 7317449157314043904. Current Snapshot: MvccSnapshot[applied={T|T < 7317449157553045504}]
I20260811 22:16:50.750347 7274 mvcc.cc:205] Tried to move back new op lower bound from 7317449157553045504 to 7317449157314043904. Current Snapshot: MvccSnapshot[applied={T|T < 7317449157553045504}]
I20260811 22:16:50.755668 7276 mvcc.cc:205] Tried to move back new op lower bound from 7317449157553045504 to 7317449157314043904. Current Snapshot: MvccSnapshot[applied={T|T < 7317449157553045504}]
I20260811 22:16:50.763669 6611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:16:50.763909 6611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37572:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:16:50.767784 6611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 906ba0b77b914addba37fafdbb07bc1d: Sending DeleteTablet for 3 replicas of tablet 4996740edf234ee492643b15d461fae7
I20260811 22:16:50.769047 6960 tablet_service.cc:1514] Processing DeleteTablet for tablet 4996740edf234ee492643b15d461fae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:50 UTC) from {username='slave'} at 127.0.0.1:53750
I20260811 22:16:50.769076 6779 tablet_service.cc:1514] Processing DeleteTablet for tablet 4996740edf234ee492643b15d461fae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:50 UTC) from {username='slave'} at 127.0.0.1:40628
I20260811 22:16:50.769631 7137 tablet_service.cc:1514] Processing DeleteTablet for tablet 4996740edf234ee492643b15d461fae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:50 UTC) from {username='slave'} at 127.0.0.1:41098
I20260811 22:16:50.771298 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 6692
I20260811 22:16:50.780668 7293 tablet_replica.cc:330] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: stopping tablet replica
I20260811 22:16:50.781232 7293 raft_consensus.cc:2229] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:16:50.782224 7293 raft_consensus.cc:2258] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:50.785085 7293 ts_tablet_manager.cc:1905] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:50.793960 7293 ts_tablet_manager.cc:1918] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:50.794240 7293 log.cc:1196] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/4996740edf234ee492643b15d461fae7
I20260811 22:16:50.794998 7293 ts_tablet_manager.cc:1939] T 4996740edf234ee492643b15d461fae7 P 945073fe636d43b18a08437c312aae29: Deleting consensus metadata
I20260811 22:16:50.802309 7292 tablet_replica.cc:330] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: stopping tablet replica
I20260811 22:16:50.802843 7292 raft_consensus.cc:2229] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:16:50.805425 7292 raft_consensus.cc:2258] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:50.807670 6589 catalog_manager.cc:4841] TS 945073fe636d43b18a08437c312aae29 (127.10.173.131:43791): tablet 4996740edf234ee492643b15d461fae7 (table test-table [id=e0a28edfce6f435e90657b421c27f1bd]) successfully deleted
I20260811 22:16:50.808070 7254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.773057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40632 (local address 127.10.173.129:40949)
0811 22:16:50.773308 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:50.773318 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:50.796929 (+ 23611us) server_negotiation.cc:407] Connection header received
0811 22:16:50.797512 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:50.797519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:50.797666 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:50.797754 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:50.798594 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.799819 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.803147 (+ 3328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.803885 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.803983 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.806242 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:50.806254 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:50.806281 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:50.807089 (+ 808us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:50.807171 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:50.807197 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:50.807310 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:50.807559 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 22:16:50.807690 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:16:50.809196 6863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:50.818490 7292 ts_tablet_manager.cc:1905] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:50.828873 7292 ts_tablet_manager.cc:1918] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:50.829152 7292 log.cc:1196] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/4996740edf234ee492643b15d461fae7
I20260811 22:16:50.829902 7292 ts_tablet_manager.cc:1939] T 4996740edf234ee492643b15d461fae7 P 703d7428a2434cdf892d6e4dc0a7040a: Deleting consensus metadata
I20260811 22:16:50.832106 6590 catalog_manager.cc:4841] TS 703d7428a2434cdf892d6e4dc0a7040a (127.10.173.130:34465): tablet 4996740edf234ee492643b15d461fae7 (table test-table [id=e0a28edfce6f435e90657b421c27f1bd]) successfully deleted
I20260811 22:16:50.915735 7291 tablet_replica.cc:330] T 4996740edf234ee492643b15d461fae7 P 245cb2753dd9444bac14b1f9cbd4201c: stopping tablet replica
I20260811 22:16:50.915896 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 6692
I20260811 22:16:50.938246 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 6875
W20260811 22:16:50.938246 6589 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.173.129:40949 (error 108)
I20260811 22:16:50.941068 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.939931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40949 (local address 127.0.0.1:40640)
0811 22:16:50.940721 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:50.940867 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:40949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":536,"threads_started":1}
W20260811 22:16:50.941650 6589 catalog_manager.cc:4584] TS 245cb2753dd9444bac14b1f9cbd4201c (127.10.173.129:40949): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4996740edf234ee492643b15d461fae7: Network error: Client connection negotiation failed: client connection to 127.10.173.129:40949: connect: Connection refused (error 111)
I20260811 22:16:50.948895 7251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.939486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.10.173.130:34465)
0811 22:16:50.939658 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:50.939663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:50.939682 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:16:50.939746 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:50.939750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:50.939828 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:50.939952 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:50.940397 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.941670 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.943996 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:50.944871 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:50.945024 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:50.946924 (+ 1900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:50.946937 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:50.946959 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:50.947976 (+ 1017us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:50.948064 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:50.948113 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:50.948224 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:50.948376 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 22:16:50.948489 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:16:50.951140 7037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:50.995666 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.995067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40949 (local address 127.0.0.1:40648)
0811 22:16:50.995330 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:50.995451 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:40949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 22:16:50.996789 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:50.996381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40949 (local address 127.0.0.1:40662)
0811 22:16:50.996539 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:50.996625 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:40949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 22:16:51.057525 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:51.056928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40949 (local address 127.0.0.1:40664)
0811 22:16:51.057244 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:51.057353 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:40949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 22:16:51.058776 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:51.058334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40949 (local address 127.0.0.1:40670)
0811 22:16:51.058530 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:51.058617 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:40949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 22:16:51.074903 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 6875
I20260811 22:16:51.104923 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 7049
I20260811 22:16:51.111960 7252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:51.105809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.10.173.131:43791)
0811 22:16:51.106005 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:51.106009 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:51.106055 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:16:51.106246 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:51.106250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:51.106336 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:51.106477 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:51.106894 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:51.107607 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:51.108975 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:51.109357 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:51.109462 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:51.110649 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:51.110659 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:51.110676 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:51.111276 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:51.111321 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:51.111342 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:51.111433 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:51.111599 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:16:51.111708 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:16:51.112706 7220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:51.139712 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:51.139212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40949 (local address 127.0.0.1:40676)
0811 22:16:51.139434 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:51.139531 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:40949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 22:16:51.140877 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:51.140424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40949 (local address 127.0.0.1:40682)
0811 22:16:51.140652 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:51.140739 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:40949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 22:16:51.202335 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 7049
I20260811 22:16:51.226080 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 6568
I20260811 22:16:51.227329 6652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:51.335539 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:51.334681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:40949 (local address 127.0.0.1:40694)
0811 22:16:51.334922 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:51.335085 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:40949: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 22:16:51.335673 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 6568
2026-08-11T22:16:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3185 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:16:51 dist-test-slave-68xk krb5kdc[7308](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:16:51 dist-test-slave-68xk krb5kdc[7308](info): set up 2 sockets
Aug 11 22:16:51 dist-test-slave-68xk krb5kdc[7308](info): commencing operation
krb5kdc: starting...
W20260811 22:16:52.756300 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.373s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:16:52 dist-test-slave-68xk krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:16:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:16:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:16:53.093593 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:45541
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:45541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:53.440327 7330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:53.440689 7330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:53.440881 7330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:53.448212 7330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:16:53.448366 7330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:53.448454 7330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:53.448544 7330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:16:53.448611 7330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:16:53.459364 7330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:45541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:45541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:53.460790 7330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:53.462042 7330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:53.474318 7340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:53.474272 7341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:53.474363 7343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:53.475313 7330 server_base.cc:1034] running on GCE node
I20260811 22:16:53.476174 7330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:53.477582 7330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:53.478820 7330 hybrid_clock.cc:648] HybridClock initialized: now 1786486613478777 us; error 57 us; skew 500 ppm
I20260811 22:16:53.479171 7330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:53 dist-test-slave-68xk krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486613, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:53.482502 7330 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:16:53.484062 7330 webserver.cc:460] Webserver started at http://127.10.173.190:38447/ using document root <none> and password file <none>
I20260811 22:16:53.484733 7330 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:53.484839 7330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:53.485118 7330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:53.488245 7330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "1a367dc5db9a4565811c06efba33d212"
format_stamp: "Formatted at 2026-08-11 22:16:53 on dist-test-slave-68xk"
server_key: "22c6d5595d137022a1ba94b088e9d9fa"
server_key_iv: "277f2546561b3c95da1bce19315f5819"
server_key_version: "encryptionkey@0"
I20260811 22:16:53.488987 7330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "1a367dc5db9a4565811c06efba33d212"
format_stamp: "Formatted at 2026-08-11 22:16:53 on dist-test-slave-68xk"
server_key: "22c6d5595d137022a1ba94b088e9d9fa"
server_key_iv: "277f2546561b3c95da1bce19315f5819"
server_key_version: "encryptionkey@0"
I20260811 22:16:53.494491 7330 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:16:53.498296 7354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:53.499838 7330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:16:53.500047 7330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "1a367dc5db9a4565811c06efba33d212"
format_stamp: "Formatted at 2026-08-11 22:16:53 on dist-test-slave-68xk"
server_key: "22c6d5595d137022a1ba94b088e9d9fa"
server_key_iv: "277f2546561b3c95da1bce19315f5819"
server_key_version: "encryptionkey@0"
I20260811 22:16:53.500280 7330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:53.523022 7330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:53.527320 7330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:53.527632 7330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:53.547199 7421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:45541 every 8 connection(s)
I20260811 22:16:53.547209 7330 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:45541
I20260811 22:16:53.548825 7330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:16:53.553053 7425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:53.554610 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 7330
I20260811 22:16:53.554838 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:16:53.555366 10934 external_mini_cluster.cc:1428] Setting key 08ecff7377395a088b90be9aa2c3f3d0
I20260811 22:16:53.567915 7425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212: Bootstrap starting.
Aug 11 22:16:53 dist-test-slave-68xk krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:53.572053 7425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:53.573238 7425 log.cc:824] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:53.577533 7425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212: No bootstrap required, opened a new log
I20260811 22:16:53.582382 7429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:53.557482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.10.173.190:45541)
0811 22:16:53.558390 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:53.558402 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:53.558510 (+ 108us) server_negotiation.cc:407] Connection header received
0811 22:16:53.560294 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:53.560322 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:53.560734 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:53.561066 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:53.563107 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:53.564483 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:53.566533 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:53.567226 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:53.567362 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:53.572630 (+ 5268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:53.572638 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:53.572654 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:53.572692 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:53.577255 (+ 4563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:53.578149 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:53.578159 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:53.578216 (+ 57us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:53.578490 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:53.578889 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:53.578900 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:53.578903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:53.579680 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:53.580005 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:53.580431 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 22:16:53.580923 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":423,"threads_started":1}
I20260811 22:16:53.587818 7425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a367dc5db9a4565811c06efba33d212" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45541 } }
I20260811 22:16:53.588245 7425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:53.588529 7425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a367dc5db9a4565811c06efba33d212, State: Initialized, Role: FOLLOWER
I20260811 22:16:53.589263 7425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a367dc5db9a4565811c06efba33d212" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45541 } }
I20260811 22:16:53.589493 7425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:16:53.589600 7425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:16:53.589843 7425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:53.593109 7425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a367dc5db9a4565811c06efba33d212" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45541 } }
I20260811 22:16:53.593766 7425 leader_election.cc:302] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a367dc5db9a4565811c06efba33d212; no voters:
I20260811 22:16:53.594504 7425 leader_election.cc:288] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:16:53.594661 7432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:16:53.595497 7432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [term 1 LEADER]: Becoming Leader. State: Replica: 1a367dc5db9a4565811c06efba33d212, State: Running, Role: LEADER
I20260811 22:16:53.596226 7432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a367dc5db9a4565811c06efba33d212" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45541 } }
I20260811 22:16:53.596973 7425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:16:53.599184 7434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a367dc5db9a4565811c06efba33d212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a367dc5db9a4565811c06efba33d212" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45541 } } }
I20260811 22:16:53.599493 7435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a367dc5db9a4565811c06efba33d212. Latest consensus state: current_term: 1 leader_uuid: "1a367dc5db9a4565811c06efba33d212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a367dc5db9a4565811c06efba33d212" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45541 } } }
I20260811 22:16:53.599742 7434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:53.599821 7435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212 [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:53.601969 7441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:16:53.608480 7441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:16:53.618360 7441 catalog_manager.cc:1347] Generated new cluster ID: eb3491848bab4dc29336553c861d14d1
I20260811 22:16:53.618530 7441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:16:53.645882 7441 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:16:53.647353 7441 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:16:53.662586 7441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212: Generated new TSK 0
I20260811 22:16:53.663798 7441 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:16:53.752303 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45541
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:54.106160 7460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:54.106499 7460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:54.106695 7460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:54.114203 7460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:54.114313 7460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:54.114540 7460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:16:54.124260 7460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:54.125527 7460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:54.126956 7460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:54.140132 7465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:54.140185 7466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:54.140093 7469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:54.141263 7460 server_base.cc:1034] running on GCE node
I20260811 22:16:54.141827 7460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:54.142827 7460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:54.144057 7460 hybrid_clock.cc:648] HybridClock initialized: now 1786486614144008 us; error 45 us; skew 500 ppm
I20260811 22:16:54.144369 7460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:54 dist-test-slave-68xk krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:54.148113 7460 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:16:54.149652 7460 webserver.cc:460] Webserver started at http://127.10.173.129:34033/ using document root <none> and password file <none>
I20260811 22:16:54.150352 7460 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:54.150543 7460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:54.150893 7460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:54.153800 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "7ba5cadd8ea247feb460930f5ebb7085"
format_stamp: "Formatted at 2026-08-11 22:16:54 on dist-test-slave-68xk"
server_key: "19ac1d9715e011ccffb054a362a7a033"
server_key_iv: "e7c7246fef98798ab9cf8c2f0001a29c"
server_key_version: "encryptionkey@0"
I20260811 22:16:54.154649 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "7ba5cadd8ea247feb460930f5ebb7085"
format_stamp: "Formatted at 2026-08-11 22:16:54 on dist-test-slave-68xk"
server_key: "19ac1d9715e011ccffb054a362a7a033"
server_key_iv: "e7c7246fef98798ab9cf8c2f0001a29c"
server_key_version: "encryptionkey@0"
I20260811 22:16:54.159883 7460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 22:16:54.163581 7477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:54.164943 7460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:16:54.165138 7460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "7ba5cadd8ea247feb460930f5ebb7085"
format_stamp: "Formatted at 2026-08-11 22:16:54 on dist-test-slave-68xk"
server_key: "19ac1d9715e011ccffb054a362a7a033"
server_key_iv: "e7c7246fef98798ab9cf8c2f0001a29c"
server_key_version: "encryptionkey@0"
I20260811 22:16:54.165314 7460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:54.190335 7460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:54.195092 7460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:54.195391 7460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:54.196367 7460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:54.197790 7460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:54.197896 7460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:54.198037 7460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:54.198109 7460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:54.231505 7460 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:36879
I20260811 22:16:54.231560 7630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:36879 every 8 connection(s)
I20260811 22:16:54.232805 7460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:16:54.236467 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 7460
I20260811 22:16:54.236701 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:16:54.237315 10934 external_mini_cluster.cc:1428] Setting key 338637bd3fca3be6d59a7e89488d8a19
Aug 11 22:16:54 dist-test-slave-68xk krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:54.255640 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:54.235445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:36021 (local address 127.10.173.190:45541)
0811 22:16:54.235940 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:54.235944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:54.237064 (+ 1120us) server_negotiation.cc:407] Connection header received
0811 22:16:54.239365 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:54.239371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:54.239442 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:54.239536 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:54.241671 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:54.242460 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:54.244913 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:54.245344 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:54.245445 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:54.249691 (+ 4246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:54.249697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:54.249700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:54.249731 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:54.252443 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:54.253196 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:54.253203 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:54.253206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:54.253337 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:54.253793 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:54.253799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:54.253802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:54.254073 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:54.254247 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:54.255023 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 22:16:54.255268 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":256,"threads_started":1}
I20260811 22:16:54.256611 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:54.235989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45541 (local address 127.10.173.129:36021)
0811 22:16:54.236857 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:54.236918 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:54.239066 (+ 2148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:54.239820 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:54.239831 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:54.240652 (+ 821us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:54.241484 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:54.241510 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:54.242641 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:54.242649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:54.244749 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:54.244767 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:54.245576 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:54.245585 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:54.245905 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:54.246725 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:16:54.246760 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:54.249458 (+ 2698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:54.252602 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:54.252611 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:54.252628 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:54.253065 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:54.253454 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:54.253459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:54.253463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:54.253658 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:54.254217 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:54.254225 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:54.254633 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 22:16:54.254916 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 22:16:54.255444 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":323,"threads_started":1}
I20260811 22:16:54.258373 7631 heartbeater.cc:347] Connected to a master server at 127.10.173.190:45541
I20260811 22:16:54.258738 7631 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:54.259466 7631 heartbeater.cc:511] Master 127.10.173.190:45541 requested a full tablet report, sending...
I20260811 22:16:54.261832 7375 ts_manager.cc:194] Registered new tserver with Master: 7ba5cadd8ea247feb460930f5ebb7085 (127.10.173.129:36879)
I20260811 22:16:54.264117 7375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:36021
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:16:54.307861 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45541
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:54.653950 7639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:54.654253 7639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:54.654361 7639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:54.661724 7639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:54.661842 7639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:54.661999 7639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:16:54.672497 7639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:54.673903 7639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:54.675618 7639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:54.689973 7653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:54.689958 7649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:54.689998 7648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:54.691394 7639 server_base.cc:1034] running on GCE node
I20260811 22:16:54.692065 7639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:54.693022 7639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:54.694258 7639 hybrid_clock.cc:648] HybridClock initialized: now 1786486614694202 us; error 36 us; skew 500 ppm
I20260811 22:16:54.694638 7639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:16:54 dist-test-slave-68xk krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:54.697968 7639 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:16:54.699689 7639 webserver.cc:460] Webserver started at http://127.10.173.130:44209/ using document root <none> and password file <none>
I20260811 22:16:54.700438 7639 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:54.700599 7639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:54.700933 7639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:54.704440 7639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "89405c76e3e24e80be4c47c664e52e6c"
format_stamp: "Formatted at 2026-08-11 22:16:54 on dist-test-slave-68xk"
server_key: "d92d944d750e80e54022810d2b718556"
server_key_iv: "17e75563ab17d4111d18666c96c6d2bc"
server_key_version: "encryptionkey@0"
I20260811 22:16:54.705345 7639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "89405c76e3e24e80be4c47c664e52e6c"
format_stamp: "Formatted at 2026-08-11 22:16:54 on dist-test-slave-68xk"
server_key: "d92d944d750e80e54022810d2b718556"
server_key_iv: "17e75563ab17d4111d18666c96c6d2bc"
server_key_version: "encryptionkey@0"
I20260811 22:16:54.710867 7639 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 22:16:54.714466 7664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:54.715879 7639 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:16:54.716077 7639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "89405c76e3e24e80be4c47c664e52e6c"
format_stamp: "Formatted at 2026-08-11 22:16:54 on dist-test-slave-68xk"
server_key: "d92d944d750e80e54022810d2b718556"
server_key_iv: "17e75563ab17d4111d18666c96c6d2bc"
server_key_version: "encryptionkey@0"
I20260811 22:16:54.716289 7639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:54.736202 7639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:54.740631 7639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:54.740960 7639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:54.741966 7639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:54.743392 7639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:54.743503 7639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:54.743666 7639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:54.743743 7639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:54.780164 7639 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:45791
I20260811 22:16:54.780268 7833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:45791 every 8 connection(s)
I20260811 22:16:54.781517 7639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:16:54.790782 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 7639
I20260811 22:16:54.790975 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:16:54.791443 10934 external_mini_cluster.cc:1428] Setting key f307be675f24aacf6a08ab27015baf7c
Aug 11 22:16:54 dist-test-slave-68xk krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486614, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:54.804593 7836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:54.784227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:42487 (local address 127.10.173.190:45541)
0811 22:16:54.784669 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:54.784674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:54.785733 (+ 1059us) server_negotiation.cc:407] Connection header received
0811 22:16:54.787474 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:54.787479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:54.787549 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:54.787654 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:54.789098 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:54.790034 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:54.791761 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:54.792346 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:54.792458 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:54.796648 (+ 4190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:54.796664 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:54.796672 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:54.796717 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:54.800560 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:54.801446 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:54.801455 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:54.801462 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:54.801632 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:54.802222 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:54.802230 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:54.802236 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:54.802695 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:54.802877 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:54.803880 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0811 22:16:54.804141 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":234,"threads_started":1}
I20260811 22:16:54.805787 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:54.784852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45541 (local address 127.10.173.130:42487)
0811 22:16:54.785571 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:54.785607 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:54.787216 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:54.787868 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:54.787876 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:54.788381 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:54.788936 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:54.788954 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:54.790164 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:54.790167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:54.791619 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:54.791632 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:54.792574 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:54.792583 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:54.792795 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:54.793801 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:16:54.793830 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:54.796404 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:54.800728 (+ 4324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:54.800736 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:54.800750 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:54.801291 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:54.801820 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:54.801824 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:54.801827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:54.802045 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:54.802954 (+ 909us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:54.802961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:54.803363 (+ 402us) client_negotiation.cc:769] Sending connection context
0811 22:16:54.803751 (+ 388us) client_negotiation.cc:240] Negotiation successful
0811 22:16:54.804469 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":289,"threads_started":1}
I20260811 22:16:54.807549 7834 heartbeater.cc:347] Connected to a master server at 127.10.173.190:45541
I20260811 22:16:54.807940 7834 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:54.808800 7834 heartbeater.cc:511] Master 127.10.173.190:45541 requested a full tablet report, sending...
I20260811 22:16:54.810515 7375 ts_manager.cc:194] Registered new tserver with Master: 89405c76e3e24e80be4c47c664e52e6c (127.10.173.130:45791)
I20260811 22:16:54.811409 7375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:42487
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:16:54.863003 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45541
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:16:55.227236 7843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:55.227514 7843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:55.227612 7843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:55.235144 7843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:55.235252 7843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:55.235424 7843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:16:55.245101 7843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:45541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:55.246471 7843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:55.247767 7843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:55.261859 7848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:55.261807 7850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:55.261909 7852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:55.262830 7843 server_base.cc:1034] running on GCE node
I20260811 22:16:55.263640 7843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:55.264797 7843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:55.266047 7843 hybrid_clock.cc:648] HybridClock initialized: now 1786486615266001 us; error 43 us; skew 500 ppm
I20260811 22:16:55.266523 7843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:16:55.267128 7631 heartbeater.cc:503] Master 127.10.173.190:45541 was elected leader, sending a full tablet report...
Aug 11 22:16:55 dist-test-slave-68xk krb5kdc[7308](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486615, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:16:55.270514 7843 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:16:55.271942 7843 webserver.cc:460] Webserver started at http://127.10.173.131:46393/ using document root <none> and password file <none>
I20260811 22:16:55.272642 7843 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:55.272781 7843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:55.273115 7843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:55.276119 7843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "80fbc36eeccd4c0698434ca23a4e73ba"
format_stamp: "Formatted at 2026-08-11 22:16:55 on dist-test-slave-68xk"
server_key: "33021072b3c877c144be772e6d84c04c"
server_key_iv: "7af6555c2bfe31df987d0b67b1ca0f7f"
server_key_version: "encryptionkey@0"
I20260811 22:16:55.276889 7843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "80fbc36eeccd4c0698434ca23a4e73ba"
format_stamp: "Formatted at 2026-08-11 22:16:55 on dist-test-slave-68xk"
server_key: "33021072b3c877c144be772e6d84c04c"
server_key_iv: "7af6555c2bfe31df987d0b67b1ca0f7f"
server_key_version: "encryptionkey@0"
I20260811 22:16:55.282253 7843 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 22:16:55.286216 7862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:55.287747 7843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:16:55.287937 7843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "80fbc36eeccd4c0698434ca23a4e73ba"
format_stamp: "Formatted at 2026-08-11 22:16:55 on dist-test-slave-68xk"
server_key: "33021072b3c877c144be772e6d84c04c"
server_key_iv: "7af6555c2bfe31df987d0b67b1ca0f7f"
server_key_version: "encryptionkey@0"
I20260811 22:16:55.288173 7843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:55.300228 7843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:55.305028 7843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:55.305404 7843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:55.306499 7843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:55.307904 7843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:55.307978 7843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:55.308151 7843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:55.308220 7843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:55.342845 7843 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:33013
I20260811 22:16:55.342895 8013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:33013 every 8 connection(s)
I20260811 22:16:55.344257 7843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:16:55.351357 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 7843
I20260811 22:16:55.351567 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:16:55.352002 10934 external_mini_cluster.cc:1428] Setting key 19283a5899e25deb6e945d0447aeea66
Aug 11 22:16:55 dist-test-slave-68xk krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486615, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:16:55.365428 8017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.347118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:40149 (local address 127.10.173.190:45541)
0811 22:16:55.347606 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:55.347611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:55.348534 (+ 923us) server_negotiation.cc:407] Connection header received
0811 22:16:55.350405 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:55.350431 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:55.350531 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:55.350656 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:55.352410 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.353267 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.355045 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.355459 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.355600 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.359242 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:55.359247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:55.359251 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:55.359289 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:55.362057 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:55.362920 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:55.362927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:55.362931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:55.363062 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:55.363511 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:55.363516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:55.363520 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:55.363897 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:55.364057 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:55.364772 (+ 715us) server_negotiation.cc:299] Negotiation successful
0811 22:16:55.365013 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":244,"threads_started":1}
I20260811 22:16:55.366667 8018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.347658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:45541 (local address 127.10.173.131:40149)
0811 22:16:55.348359 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:55.348401 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:55.350118 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:55.350921 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:55.350931 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:55.351519 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:55.352230 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.352250 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.353439 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.353450 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.354879 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.354901 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.355691 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.355700 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.355971 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.356704 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:16:55.356726 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:55.359014 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:55.362219 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:55.362228 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:55.362248 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:55.362793 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:55.363180 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:16:55.363186 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:16:55.363189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:16:55.363383 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:16:55.364031 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:55.364040 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:55.364391 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 22:16:55.364660 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 22:16:55.365229 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":274,"threads_started":1}
I20260811 22:16:55.368259 8014 heartbeater.cc:347] Connected to a master server at 127.10.173.190:45541
I20260811 22:16:55.368662 8014 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:55.369388 8014 heartbeater.cc:511] Master 127.10.173.190:45541 requested a full tablet report, sending...
I20260811 22:16:55.370923 7375 ts_manager.cc:194] Registered new tserver with Master: 80fbc36eeccd4c0698434ca23a4e73ba (127.10.173.131:33013)
I20260811 22:16:55.371721 7375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:40149
I20260811 22:16:55.379990 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:16:55.398021 8017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.384249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.10.173.190:45541)
0811 22:16:55.384429 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:55.384434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:55.384792 (+ 358us) server_negotiation.cc:407] Connection header received
0811 22:16:55.384930 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:55.384933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:55.385014 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:55.385107 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:55.386597 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.387343 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.388683 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.389095 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.389189 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.392309 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:55.392315 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:55.392319 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:55.392356 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:55.395176 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:55.395977 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:55.395981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:55.395984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:55.396164 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:55.396620 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:55.396624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:55.396626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:55.396956 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:55.397293 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:55.397640 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 22:16:55.397789 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:16:55.402770 7375 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:16:55.410861 7375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:16:55.436683 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.425215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53708 (local address 127.10.173.129:36879)
0811 22:16:55.428014 (+ 2799us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:55.428022 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:55.428049 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:16:55.428148 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:55.428155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:55.428717 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:55.429745 (+ 1028us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:55.429959 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.431367 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.434329 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.435441 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.435682 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.435878 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:55.436122 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 22:16:55.436291 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2615,"thread_start_us":2440,"threads_started":1}
I20260811 22:16:55.436782 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.424814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:36879 (local address 127.0.0.1:53708)
0811 22:16:55.425812 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:55.425847 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:55.425992 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:55.428944 (+ 2952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:55.428949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:55.428976 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:55.429395 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.429405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.431524 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.431533 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.434122 (+ 2589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.434135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.435659 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.435668 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.435850 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.435903 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 22:16:55.436079 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 22:16:55.436294 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":55,"thread_start_us":429,"threads_started":1}
I20260811 22:16:55.437880 8031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.426256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:45791 (local address 127.0.0.1:59862)
0811 22:16:55.426915 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:55.426939 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:55.427058 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:55.428611 (+ 1553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:55.428623 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:55.428663 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:55.429366 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.429387 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.431581 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.431586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.434481 (+ 2895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.434496 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.436634 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.436641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.437227 (+ 586us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.437262 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:16:55.437352 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:16:55.437482 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":147,"thread_start_us":328,"threads_started":1}
I20260811 22:16:55.438555 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.426541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59862 (local address 127.10.173.130:45791)
0811 22:16:55.427284 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:55.427298 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:55.427338 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:16:55.427477 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:55.427488 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:55.427789 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:55.428012 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:55.429874 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.431378 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.434657 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.436140 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.436314 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.436493 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:55.438022 (+ 1529us) server_negotiation.cc:299] Negotiation successful
0811 22:16:55.438211 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":368,"threads_started":1}
I20260811 22:16:55.439744 8033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.427690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:33013 (local address 127.0.0.1:35080)
0811 22:16:55.428286 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:55.428305 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:55.428425 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:55.431566 (+ 3141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:55.431570 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:55.431605 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:55.432160 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.432207 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.434264 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.434269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.436444 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.436461 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.438902 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.438914 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.439103 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.439135 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:16:55.439246 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:16:55.439363 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":33,"thread_start_us":718,"threads_started":1}
I20260811 22:16:55.439827 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.428346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35080 (local address 127.10.173.131:33013)
0811 22:16:55.430962 (+ 2616us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:55.430971 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:55.431002 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:16:55.431095 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:55.431102 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:55.431324 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:55.431529 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:55.432464 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.434114 (+ 1650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.436639 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.438403 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.438637 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.438821 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:55.439327 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 22:16:55.439492 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2436,"thread_start_us":2274,"threads_started":1}
I20260811 22:16:55.440943 7542 tablet_service.cc:1467] Processing CreateTablet for tablet e59f467606194f88858e2c86357d22c9 (DEFAULT_TABLE table=test-table [id=5c50f17165e746419895b8e206eeefe0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:55.441811 7728 tablet_service.cc:1467] Processing CreateTablet for tablet e59f467606194f88858e2c86357d22c9 (DEFAULT_TABLE table=test-table [id=5c50f17165e746419895b8e206eeefe0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:55.442530 7542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e59f467606194f88858e2c86357d22c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:55.444839 7930 tablet_service.cc:1467] Processing CreateTablet for tablet e59f467606194f88858e2c86357d22c9 (DEFAULT_TABLE table=test-table [id=5c50f17165e746419895b8e206eeefe0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:55.445779 7728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e59f467606194f88858e2c86357d22c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:55.446151 7930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e59f467606194f88858e2c86357d22c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:55.454267 8035 tablet_bootstrap.cc:492] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: Bootstrap starting.
I20260811 22:16:55.458380 8035 tablet_bootstrap.cc:654] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:55.459528 8035 log.cc:824] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:55.461714 8037 tablet_bootstrap.cc:492] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: Bootstrap starting.
I20260811 22:16:55.463393 8038 tablet_bootstrap.cc:492] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: Bootstrap starting.
I20260811 22:16:55.466777 8037 tablet_bootstrap.cc:654] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:55.466849 8038 tablet_bootstrap.cc:654] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:55.468034 8038 log.cc:824] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:55.468060 8037 log.cc:824] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:55.469650 8035 tablet_bootstrap.cc:492] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: No bootstrap required, opened a new log
I20260811 22:16:55.469969 8035 ts_tablet_manager.cc:1397] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260811 22:16:55.471259 8038 tablet_bootstrap.cc:492] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: No bootstrap required, opened a new log
I20260811 22:16:55.471547 8038 ts_tablet_manager.cc:1397] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 22:16:55.472568 8037 tablet_bootstrap.cc:492] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: No bootstrap required, opened a new log
I20260811 22:16:55.472849 8037 ts_tablet_manager.cc:1397] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 22:16:55.478277 8035 raft_consensus.cc:348] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba5cadd8ea247feb460930f5ebb7085" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36879 } } peers { permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 } } peers { permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 } }
I20260811 22:16:55.478736 8035 raft_consensus.cc:374] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:55.478843 8035 raft_consensus.cc:729] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba5cadd8ea247feb460930f5ebb7085, State: Initialized, Role: FOLLOWER
I20260811 22:16:55.479480 8035 consensus_queue.cc:260] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba5cadd8ea247feb460930f5ebb7085" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36879 } } peers { permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 } } peers { permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 } }
I20260811 22:16:55.481412 8035 ts_tablet_manager.cc:1428] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: Time spent starting tablet: real 0.011s user 0.013s sys 0.000s
I20260811 22:16:55.482443 8038 raft_consensus.cc:348] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba5cadd8ea247feb460930f5ebb7085" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36879 } } peers { permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 } } peers { permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 } }
I20260811 22:16:55.482892 8038 raft_consensus.cc:374] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:55.482991 8038 raft_consensus.cc:729] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80fbc36eeccd4c0698434ca23a4e73ba, State: Initialized, Role: FOLLOWER
I20260811 22:16:55.483610 8038 consensus_queue.cc:260] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba5cadd8ea247feb460930f5ebb7085" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36879 } } peers { permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 } } peers { permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 } }
I20260811 22:16:55.483584 8037 raft_consensus.cc:348] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba5cadd8ea247feb460930f5ebb7085" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36879 } } peers { permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 } } peers { permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 } }
I20260811 22:16:55.484030 8037 raft_consensus.cc:374] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:55.484165 8037 raft_consensus.cc:729] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89405c76e3e24e80be4c47c664e52e6c, State: Initialized, Role: FOLLOWER
I20260811 22:16:55.484889 8037 consensus_queue.cc:260] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba5cadd8ea247feb460930f5ebb7085" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36879 } } peers { permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 } } peers { permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 } }
W20260811 22:16:55.485311 7632 tablet.cc:2369] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:16:55.486269 8038 ts_tablet_manager.cc:1428] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260811 22:16:55.486888 8014 heartbeater.cc:503] Master 127.10.173.190:45541 was elected leader, sending a full tablet report...
I20260811 22:16:55.487536 7834 heartbeater.cc:503] Master 127.10.173.190:45541 was elected leader, sending a full tablet report...
I20260811 22:16:55.488246 8037 ts_tablet_manager.cc:1428] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
W20260811 22:16:55.533613 7835 tablet.cc:2369] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:16:55.596093 8016 tablet.cc:2369] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:16:55.640170 8042 raft_consensus.cc:482] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:16:55.640545 8042 raft_consensus.cc:504] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba5cadd8ea247feb460930f5ebb7085" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36879 } } peers { permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 } } peers { permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 } }
I20260811 22:16:55.642521 8042 leader_election.cc:288] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89405c76e3e24e80be4c47c664e52e6c (127.10.173.130:45791), 80fbc36eeccd4c0698434ca23a4e73ba (127.10.173.131:33013)
I20260811 22:16:55.650012 8034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.643170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:38611 (local address 127.10.173.131:33013)
0811 22:16:55.643451 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:55.643461 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:55.644396 (+ 935us) server_negotiation.cc:407] Connection header received
0811 22:16:55.644614 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:55.644619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:55.644681 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:55.644784 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:55.645478 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.646229 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.648240 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.649028 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.649221 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.649309 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:55.649621 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 22:16:55.649747 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:16:55.650056 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.643080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:39961 (local address 127.10.173.130:45791)
0811 22:16:55.643347 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:55.643356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:55.644299 (+ 943us) server_negotiation.cc:407] Connection header received
0811 22:16:55.644529 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:55.644535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:55.644598 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:55.644697 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:55.645394 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.646150 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.648262 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.649047 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.649227 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.649325 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:55.649650 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 22:16:55.649779 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:16:55.650200 8046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.642934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:45791 (local address 127.10.173.129:39961)
0811 22:16:55.644167 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:55.644184 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:55.644369 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:55.644714 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:55.644718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:55.644741 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:55.645227 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.645239 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.646292 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.646300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.648081 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.648095 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.649325 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.649332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.649485 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.649508 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:16:55.649617 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:16:55.649745 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1079,"mutex_wait_us":72,"spinlock_wait_cycles":9600,"thread_start_us":735,"threads_started":1}
I20260811 22:16:55.650647 8048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.643031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:33013 (local address 127.10.173.129:38611)
0811 22:16:55.644274 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:55.644289 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:55.644454 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:55.644804 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:55.644812 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:55.644868 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:55.645303 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.645312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.646383 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.646387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.648052 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:55.648068 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.649276 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.649283 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:55.649443 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.649469 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:16:55.649583 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:16:55.649716 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1104,"mutex_wait_us":3,"spinlock_wait_cycles":39040,"thread_start_us":762,"threads_started":1}
I20260811 22:16:55.651429 7758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e59f467606194f88858e2c86357d22c9" candidate_uuid: "7ba5cadd8ea247feb460930f5ebb7085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89405c76e3e24e80be4c47c664e52e6c" is_pre_election: true
I20260811 22:16:55.651677 7955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e59f467606194f88858e2c86357d22c9" candidate_uuid: "7ba5cadd8ea247feb460930f5ebb7085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" is_pre_election: true
I20260811 22:16:55.652017 7758 raft_consensus.cc:2454] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba5cadd8ea247feb460930f5ebb7085 in term 0.
I20260811 22:16:55.652240 7955 raft_consensus.cc:2454] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba5cadd8ea247feb460930f5ebb7085 in term 0.
I20260811 22:16:55.652766 7478 leader_election.cc:302] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba5cadd8ea247feb460930f5ebb7085, 89405c76e3e24e80be4c47c664e52e6c; no voters:
I20260811 22:16:55.653244 8042 raft_consensus.cc:2781] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:16:55.653394 8042 raft_consensus.cc:482] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:16:55.653522 8042 raft_consensus.cc:3037] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:55.655915 8042 raft_consensus.cc:504] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba5cadd8ea247feb460930f5ebb7085" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36879 } } peers { permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 } } peers { permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 } }
I20260811 22:16:55.656615 8042 leader_election.cc:288] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [CANDIDATE]: Term 1 election: Requested vote from peers 89405c76e3e24e80be4c47c664e52e6c (127.10.173.130:45791), 80fbc36eeccd4c0698434ca23a4e73ba (127.10.173.131:33013)
I20260811 22:16:55.657106 7955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e59f467606194f88858e2c86357d22c9" candidate_uuid: "7ba5cadd8ea247feb460930f5ebb7085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80fbc36eeccd4c0698434ca23a4e73ba"
I20260811 22:16:55.657121 7758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e59f467606194f88858e2c86357d22c9" candidate_uuid: "7ba5cadd8ea247feb460930f5ebb7085" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89405c76e3e24e80be4c47c664e52e6c"
I20260811 22:16:55.657332 7955 raft_consensus.cc:3037] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:55.657357 7758 raft_consensus.cc:3037] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:55.659663 7955 raft_consensus.cc:2454] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba5cadd8ea247feb460930f5ebb7085 in term 1.
I20260811 22:16:55.660066 7758 raft_consensus.cc:2454] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba5cadd8ea247feb460930f5ebb7085 in term 1.
I20260811 22:16:55.660146 7483 leader_election.cc:302] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ba5cadd8ea247feb460930f5ebb7085, 80fbc36eeccd4c0698434ca23a4e73ba; no voters:
I20260811 22:16:55.660530 8042 raft_consensus.cc:2781] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:16:55.661047 8042 raft_consensus.cc:686] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 1 LEADER]: Becoming Leader. State: Replica: 7ba5cadd8ea247feb460930f5ebb7085, State: Running, Role: LEADER
I20260811 22:16:55.661546 8042 consensus_queue.cc:237] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba5cadd8ea247feb460930f5ebb7085" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36879 } } peers { permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 } } peers { permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 } }
I20260811 22:16:55.666733 7375 catalog_manager.cc:5495] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ba5cadd8ea247feb460930f5ebb7085 (127.10.173.129). New cstate: current_term: 1 leader_uuid: "7ba5cadd8ea247feb460930f5ebb7085" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba5cadd8ea247feb460930f5ebb7085" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 36879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 } health_report { overall_health: UNKNOWN } } }
I20260811 22:16:55.736511 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.730878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53720 (local address 127.10.173.129:36879)
0811 22:16:55.731139 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:55.731144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:55.731167 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:16:55.731252 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:55.731255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:55.731341 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:55.731469 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:16:55.731869 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.732565 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.734089 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.734663 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.734779 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.735298 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:16:55.735853 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:16:55.735996 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:55.736160 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 22:16:55.736214 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:16:55.748164 7955 raft_consensus.cc:1261] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba [term 1 FOLLOWER]: Refusing update from remote peer 7ba5cadd8ea247feb460930f5ebb7085: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:16:55.749508 8042 consensus_queue.cc:1035] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80fbc36eeccd4c0698434ca23a4e73ba" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 33013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:16:55.752009 7758 raft_consensus.cc:1261] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c [term 1 FOLLOWER]: Refusing update from remote peer 7ba5cadd8ea247feb460930f5ebb7085: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:16:55.752969 8042 consensus_queue.cc:1035] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89405c76e3e24e80be4c47c664e52e6c" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:16:55.762445 8053 mvcc.cc:205] Tried to move back new op lower bound from 7317449178083405824 to 7317449177753759744. Current Snapshot: MvccSnapshot[applied={T|T < 7317449178083405824}]
I20260811 22:16:55.768764 8054 mvcc.cc:205] Tried to move back new op lower bound from 7317449178083405824 to 7317449177753759744. Current Snapshot: MvccSnapshot[applied={T|T < 7317449178083405824}]
I20260811 22:16:55.781626 8056 mvcc.cc:205] Tried to move back new op lower bound from 7317449178083405824 to 7317449177753759744. Current Snapshot: MvccSnapshot[applied={T|T < 7317449178083405824 or (T in {7317449178083405824})}]
I20260811 22:16:55.796126 7374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:16:55.796452 7374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:16:55.800602 7374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a367dc5db9a4565811c06efba33d212: Sending DeleteTablet for 3 replicas of tablet e59f467606194f88858e2c86357d22c9
I20260811 22:16:55.801939 7728 tablet_service.cc:1514] Processing DeleteTablet for tablet e59f467606194f88858e2c86357d22c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:55 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:59862
I20260811 22:16:55.801934 7930 tablet_service.cc:1514] Processing DeleteTablet for tablet e59f467606194f88858e2c86357d22c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:55 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:35080
I20260811 22:16:55.803134 7542 tablet_service.cc:1514] Processing DeleteTablet for tablet e59f467606194f88858e2c86357d22c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:55 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:53708
I20260811 22:16:55.806041 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 7460
I20260811 22:16:55.843222 8068 tablet_replica.cc:330] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: stopping tablet replica
I20260811 22:16:55.844077 8068 raft_consensus.cc:2229] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:16:55.845191 8068 raft_consensus.cc:2258] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:55.851267 8065 tablet_replica.cc:330] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: stopping tablet replica
I20260811 22:16:55.852313 8065 raft_consensus.cc:2229] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:16:55.856914 8067 tablet_replica.cc:330] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: stopping tablet replica
I20260811 22:16:55.857362 8067 raft_consensus.cc:2229] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:16:55.857929 8067 raft_consensus.cc:2258] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:55.860371 8067 ts_tablet_manager.cc:1905] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:55.860383 8068 ts_tablet_manager.cc:1905] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:55.865204 8065 raft_consensus.cc:2258] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:55.868234 8068 ts_tablet_manager.cc:1918] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:55.868564 8068 log.cc:1196] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/e59f467606194f88858e2c86357d22c9
I20260811 22:16:55.869263 8068 ts_tablet_manager.cc:1939] T e59f467606194f88858e2c86357d22c9 P 7ba5cadd8ea247feb460930f5ebb7085: Deleting consensus metadata
I20260811 22:16:55.871397 8065 ts_tablet_manager.cc:1905] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:55.871500 7355 catalog_manager.cc:4841] TS 7ba5cadd8ea247feb460930f5ebb7085 (127.10.173.129:36879): tablet e59f467606194f88858e2c86357d22c9 (table test-table [id=5c50f17165e746419895b8e206eeefe0]) successfully deleted
I20260811 22:16:55.876746 8065 ts_tablet_manager.cc:1918] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:55.880620 8067 ts_tablet_manager.cc:1918] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 11 22:16:55 dist-test-slave-68xk krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:16:55.880962 8067 log.cc:1196] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/e59f467606194f88858e2c86357d22c9
I20260811 22:16:55.881827 8067 ts_tablet_manager.cc:1939] T e59f467606194f88858e2c86357d22c9 P 80fbc36eeccd4c0698434ca23a4e73ba: Deleting consensus metadata
I20260811 22:16:55.876942 8065 log.cc:1196] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/e59f467606194f88858e2c86357d22c9
I20260811 22:16:55.882941 8065 ts_tablet_manager.cc:1939] T e59f467606194f88858e2c86357d22c9 P 89405c76e3e24e80be4c47c664e52e6c: Deleting consensus metadata
I20260811 22:16:55.884588 7358 catalog_manager.cc:4841] TS 80fbc36eeccd4c0698434ca23a4e73ba (127.10.173.131:33013): tablet e59f467606194f88858e2c86357d22c9 (table test-table [id=5c50f17165e746419895b8e206eeefe0]) successfully deleted
I20260811 22:16:55.885241 7355 catalog_manager.cc:4841] TS 89405c76e3e24e80be4c47c664e52e6c (127.10.173.130:45791): tablet e59f467606194f88858e2c86357d22c9 (table test-table [id=5c50f17165e746419895b8e206eeefe0]) successfully deleted
I20260811 22:16:55.893741 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:55.807566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.10.173.129:36879)
0811 22:16:55.807858 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:55.807867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:55.846686 (+ 38819us) server_negotiation.cc:407] Connection header received
0811 22:16:55.847051 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:55.847060 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:55.847203 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:55.847314 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:55.852802 (+ 5488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.854481 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.868870 (+ 14389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:55.869681 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:55.869823 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:55.885720 (+ 15897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:55.885737 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:55.885754 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:55.885808 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:55.889722 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:55.890761 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:55.890769 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:55.890772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:55.890941 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:55.891420 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:55.891426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:55.891449 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:55.892046 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:55.892508 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:55.892833 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 22:16:55.893113 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 22:16:55.894647 7625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:56.012100 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 7460
I20260811 22:16:56.049733 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 7639
Aug 11 22:16:56 dist-test-slave-68xk krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:16:56.071194 8032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:56.050692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59864 (local address 127.10.173.130:45791)
0811 22:16:56.050905 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:56.050912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:56.051058 (+ 146us) server_negotiation.cc:407] Connection header received
0811 22:16:56.051260 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:56.051264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:56.051339 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:56.051430 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:56.053495 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:56.054348 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:56.056955 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:56.057435 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:56.057544 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:56.063928 (+ 6384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:56.063943 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:56.063963 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:56.064010 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:56.067691 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:56.068741 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:56.068748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:56.068751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:56.068983 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:56.069458 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:56.069462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:56.069464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:56.069950 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:56.070186 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:56.070651 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 22:16:56.070835 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:16:56.071949 7825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:56.164521 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 7639
I20260811 22:16:56.188841 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 7843
Aug 11 22:16:56 dist-test-slave-68xk krb5kdc[7308](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:16:56.205783 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:56.189707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35084 (local address 127.10.173.131:33013)
0811 22:16:56.190280 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:56.190285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:56.190304 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:16:56.190380 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:56.190384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:56.190539 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:56.190643 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:56.192175 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:56.193090 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:56.194565 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:56.195021 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:56.195171 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:56.199783 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:56.199801 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:56.199819 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:16:56.199868 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:56.203076 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:56.203709 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:56.203717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:56.203719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:56.203925 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:16:56.204329 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:16:56.204332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:16:56.204334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:16:56.204700 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:56.204949 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:56.205280 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 22:16:56.205447 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":312,"threads_started":1}
I20260811 22:16:56.206684 8009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:56.299486 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 7843
I20260811 22:16:56.324901 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 7330
I20260811 22:16:56.326078 7416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:56.440815 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 7330
2026-08-11T22:16:56Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5110 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T22:16:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:16:56Z Disabled control of system clock
I20260811 22:16:56.544780 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38487
--webserver_interface=127.10.173.190
--webserver_port=0
--builtin_ntp_servers=127.10.173.148:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:38487
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:16:56.910228 8081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:56.910604 8081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:56.910697 8081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:56.917896 8081 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:16:56.918001 8081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:56.918045 8081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:56.918097 8081 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:16:56.918147 8081 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:16:56.927443 8081 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:38487
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:56.928709 8081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:56.930523 8081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:56.942575 8093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:56.942730 8087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:56.942692 8088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:56.943681 8081 server_base.cc:1034] running on GCE node
I20260811 22:16:56.944497 8081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:56.945928 8081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:56.947170 8081 hybrid_clock.cc:648] HybridClock initialized: now 1786486616947096 us; error 35 us; skew 500 ppm
I20260811 22:16:56.947489 8081 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:16:56.950176 8081 webserver.cc:460] Webserver started at http://127.10.173.190:41929/ using document root <none> and password file <none>
I20260811 22:16:56.951152 8081 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:56.951285 8081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:56.951709 8081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:56.956275 8081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba"
format_stamp: "Formatted at 2026-08-11 22:16:56 on dist-test-slave-68xk"
server_key: "60808e544349aa876e2a6821cd13617c"
server_key_iv: "95c8a21d5a4f812f961f2e7ac95d2a16"
server_key_version: "encryptionkey@0"
I20260811 22:16:56.957520 8081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba"
format_stamp: "Formatted at 2026-08-11 22:16:56 on dist-test-slave-68xk"
server_key: "60808e544349aa876e2a6821cd13617c"
server_key_iv: "95c8a21d5a4f812f961f2e7ac95d2a16"
server_key_version: "encryptionkey@0"
I20260811 22:16:56.967430 8081 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.000s
I20260811 22:16:56.973084 8099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:56.975302 8081 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 22:16:56.975512 8081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba"
format_stamp: "Formatted at 2026-08-11 22:16:56 on dist-test-slave-68xk"
server_key: "60808e544349aa876e2a6821cd13617c"
server_key_iv: "95c8a21d5a4f812f961f2e7ac95d2a16"
server_key_version: "encryptionkey@0"
I20260811 22:16:56.975808 8081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:56.998373 8081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:56.999416 8081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:56.999735 8081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:57.022130 8081 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:38487
I20260811 22:16:57.022228 8173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:38487 every 8 connection(s)
I20260811 22:16:57.023654 8081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:16:57.027709 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 8081
I20260811 22:16:57.027890 8174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:57.027915 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:16:57.028506 10934 external_mini_cluster.cc:1428] Setting key 4aaaa47e696380ad4400420be7394b56
I20260811 22:16:57.049530 8174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba: Bootstrap starting.
I20260811 22:16:57.054216 8174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:57.055572 8174 log.cc:824] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:57.056308 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:57.030365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58688 (local address 127.10.173.190:38487)
0811 22:16:57.039496 (+ 9131us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:57.039516 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:57.039597 (+ 81us) server_negotiation.cc:407] Connection header received
0811 22:16:57.043409 (+ 3812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:57.043449 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:57.044240 (+ 791us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:57.045720 (+ 1480us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:57.046430 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:57.048409 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:57.050350 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:57.051683 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:57.052715 (+ 1032us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:57.053513 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:57.053522 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:57.053536 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:57.054207 (+ 671us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:57.054341 (+ 134us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:57.054381 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:57.054541 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:57.054845 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 22:16:57.055175 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8481,"thread_start_us":7796,"threads_started":1}
I20260811 22:16:57.058907 8174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba: No bootstrap required, opened a new log
I20260811 22:16:57.066811 8174 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38487 } }
I20260811 22:16:57.067224 8174 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:57.067342 8174 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb9b3f717e0b4bb29609ac35cd29c7ba, State: Initialized, Role: FOLLOWER
I20260811 22:16:57.067984 8174 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38487 } }
I20260811 22:16:57.068255 8174 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:16:57.068399 8174 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:16:57.068583 8174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:57.071058 8174 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38487 } }
I20260811 22:16:57.071653 8174 leader_election.cc:302] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb9b3f717e0b4bb29609ac35cd29c7ba; no voters:
I20260811 22:16:57.072430 8174 leader_election.cc:288] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:16:57.072567 8181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:16:57.073385 8181 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [term 1 LEADER]: Becoming Leader. State: Replica: bb9b3f717e0b4bb29609ac35cd29c7ba, State: Running, Role: LEADER
I20260811 22:16:57.073926 8181 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38487 } }
I20260811 22:16:57.074743 8174 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:16:57.077171 8182 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38487 } } }
I20260811 22:16:57.077737 8182 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:57.077309 8183 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb9b3f717e0b4bb29609ac35cd29c7ba. Latest consensus state: current_term: 1 leader_uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb9b3f717e0b4bb29609ac35cd29c7ba" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38487 } } }
I20260811 22:16:57.079671 8183 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba [sys.catalog]: This master's current role is: LEADER
I20260811 22:16:57.080250 8193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:16:57.084841 8193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:16:57.094377 8193 catalog_manager.cc:1347] Generated new cluster ID: b7d38cc997094031adefcc6c9101d148
I20260811 22:16:57.094533 8193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:16:57.114637 8193 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:16:57.116081 8193 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:16:57.124095 8193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba: Generated new TSK 0
I20260811 22:16:57.125100 8193 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:16:57.133687 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38487
--builtin_ntp_servers=127.10.173.148:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:16:57.487083 8210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:57.487375 8210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:57.487545 8210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:57.495005 8210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:57.495111 8210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:57.495272 8210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:16:57.504437 8210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:57.505810 8210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:57.507810 8210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:57.521350 8216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:57.521602 8217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:57.521781 8219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:57.522691 8210 server_base.cc:1034] running on GCE node
I20260811 22:16:57.523319 8210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:57.524365 8210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:57.525614 8210 hybrid_clock.cc:648] HybridClock initialized: now 1786486617525540 us; error 61 us; skew 500 ppm
I20260811 22:16:57.525985 8210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:16:57.528576 8210 webserver.cc:460] Webserver started at http://127.10.173.129:34715/ using document root <none> and password file <none>
I20260811 22:16:57.529320 8210 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:57.529436 8210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:57.529764 8210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:57.532774 8210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "1eeee4de17084e52935dde50712b6034"
format_stamp: "Formatted at 2026-08-11 22:16:57 on dist-test-slave-68xk"
server_key: "d0464f17744f970dd767cc3eed645959"
server_key_iv: "05d58a62bd81b8892e98d21560730d6e"
server_key_version: "encryptionkey@0"
I20260811 22:16:57.533592 8210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "1eeee4de17084e52935dde50712b6034"
format_stamp: "Formatted at 2026-08-11 22:16:57 on dist-test-slave-68xk"
server_key: "d0464f17744f970dd767cc3eed645959"
server_key_iv: "05d58a62bd81b8892e98d21560730d6e"
server_key_version: "encryptionkey@0"
I20260811 22:16:57.539089 8210 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:16:57.542814 8228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:57.544204 8210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:16:57.544401 8210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "1eeee4de17084e52935dde50712b6034"
format_stamp: "Formatted at 2026-08-11 22:16:57 on dist-test-slave-68xk"
server_key: "d0464f17744f970dd767cc3eed645959"
server_key_iv: "05d58a62bd81b8892e98d21560730d6e"
server_key_version: "encryptionkey@0"
I20260811 22:16:57.544672 8210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:57.566299 8210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:57.567278 8210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:57.567643 8210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:57.568672 8210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:57.570173 8210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:57.570262 8210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:57.570335 8210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:57.570393 8210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:57.607547 8210 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:45553
I20260811 22:16:57.607650 8378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:45553 every 8 connection(s)
I20260811 22:16:57.609318 8210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:16:57.618664 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 8210
I20260811 22:16:57.618856 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:16:57.619371 10934 external_mini_cluster.cc:1428] Setting key fa6c653d5e65bd27fd4de614c74e7373
I20260811 22:16:57.622215 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38487
--builtin_ntp_servers=127.10.173.148:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:16:57.622535 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:57.612123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:39379 (local address 127.10.173.190:38487)
0811 22:16:57.612627 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:57.612633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:57.613822 (+ 1189us) server_negotiation.cc:407] Connection header received
0811 22:16:57.615714 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:57.615721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:57.615816 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:57.615936 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:57.617021 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:57.617792 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:57.619616 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:57.620017 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:57.620168 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:57.621609 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:57.621614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:57.621617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:57.621636 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:57.621680 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:57.621692 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:57.621789 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:57.622019 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 22:16:57.622121 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":267,"threads_started":1}
I20260811 22:16:57.623859 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:57.612698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38487 (local address 127.10.173.129:39379)
0811 22:16:57.613648 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:57.613693 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:57.615431 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:57.616173 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:57.616182 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:57.616269 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:57.616842 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:57.616858 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:57.617955 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:57.617960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:57.619461 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:57.619478 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:57.620255 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:57.620265 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:57.620533 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:57.621274 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:16:57.621277 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:57.621317 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:16:57.621401 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:57.621790 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:57.621804 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:57.621814 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:16:57.622006 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 22:16:57.622418 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":292,"threads_started":1}
I20260811 22:16:57.625521 8380 heartbeater.cc:347] Connected to a master server at 127.10.173.190:38487
I20260811 22:16:57.625851 8380 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:57.626771 8380 heartbeater.cc:511] Master 127.10.173.190:38487 requested a full tablet report, sending...
I20260811 22:16:57.629166 8124 ts_manager.cc:194] Registered new tserver with Master: 1eeee4de17084e52935dde50712b6034 (127.10.173.129:45553)
I20260811 22:16:57.631394 8124 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.129:39379
W20260811 22:16:57.986444 8384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:57.986806 8384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:57.986975 8384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:57.994310 8384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:57.994483 8384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:57.994683 8384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:16:58.003816 8384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:58.005379 8384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:58.007277 8384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:58.020938 8392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:58.020968 8393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:58.021831 8384 server_base.cc:1034] running on GCE node
W20260811 22:16:58.020895 8395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:58.025453 8384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:58.026571 8384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:58.027834 8384 hybrid_clock.cc:648] HybridClock initialized: now 1786486618027760 us; error 71 us; skew 500 ppm
I20260811 22:16:58.028194 8384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:16:58.035917 8384 webserver.cc:460] Webserver started at http://127.10.173.130:45319/ using document root <none> and password file <none>
I20260811 22:16:58.036651 8384 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:58.036744 8384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:58.037115 8384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:58.040141 8384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "0776d30db1374274a82887049ea10d01"
format_stamp: "Formatted at 2026-08-11 22:16:58 on dist-test-slave-68xk"
server_key: "f71709d101adea81f0057df464a65840"
server_key_iv: "80dd614f64e2b2304b60235626ffaf19"
server_key_version: "encryptionkey@0"
I20260811 22:16:58.040921 8384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "0776d30db1374274a82887049ea10d01"
format_stamp: "Formatted at 2026-08-11 22:16:58 on dist-test-slave-68xk"
server_key: "f71709d101adea81f0057df464a65840"
server_key_iv: "80dd614f64e2b2304b60235626ffaf19"
server_key_version: "encryptionkey@0"
I20260811 22:16:58.046213 8384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 22:16:58.049890 8402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:58.051271 8384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:16:58.051442 8384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "0776d30db1374274a82887049ea10d01"
format_stamp: "Formatted at 2026-08-11 22:16:58 on dist-test-slave-68xk"
server_key: "f71709d101adea81f0057df464a65840"
server_key_iv: "80dd614f64e2b2304b60235626ffaf19"
server_key_version: "encryptionkey@0"
I20260811 22:16:58.051654 8384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:58.065480 8384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:58.066385 8384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:58.066792 8384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:58.067826 8384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:58.069339 8384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:58.069428 8384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:58.069598 8384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:58.069692 8384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:58.105868 8384 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:37681
I20260811 22:16:58.105933 8575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:37681 every 8 connection(s)
I20260811 22:16:58.107261 8384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:16:58.116757 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 8384
I20260811 22:16:58.116969 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:16:58.117468 10934 external_mini_cluster.cc:1428] Setting key dd3d23fb2b87c0abda2f57de4e8c726a
I20260811 22:16:58.121220 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38487
--builtin_ntp_servers=127.10.173.148:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:16:58.121464 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.110107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:45941 (local address 127.10.173.190:38487)
0811 22:16:58.110332 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:58.110340 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:58.111893 (+ 1553us) server_negotiation.cc:407] Connection header received
0811 22:16:58.113634 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:58.113639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:58.113706 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:58.113799 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:58.115179 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.115874 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.117999 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.118574 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.118680 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.120537 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:58.120552 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:58.120557 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:58.120579 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:58.120620 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:58.120638 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:58.120740 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:58.120999 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 22:16:58.121113 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:16:58.122503 8583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.110720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38487 (local address 127.10.173.130:45941)
0811 22:16:58.111727 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:58.111770 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:58.113372 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:58.114126 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:58.114135 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:58.114228 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:58.114971 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.114991 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.116116 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.116122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.117838 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.117853 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.118933 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.118941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.119264 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.120198 (+ 934us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:16:58.120201 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:58.120246 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:16:58.120330 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:58.120765 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:58.120774 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:58.120780 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:16:58.120941 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 22:16:58.121263 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":385,"threads_started":1}
I20260811 22:16:58.123960 8579 heartbeater.cc:347] Connected to a master server at 127.10.173.190:38487
I20260811 22:16:58.124296 8579 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:58.125277 8579 heartbeater.cc:511] Master 127.10.173.190:38487 requested a full tablet report, sending...
I20260811 22:16:58.126935 8124 ts_manager.cc:194] Registered new tserver with Master: 0776d30db1374274a82887049ea10d01 (127.10.173.130:37681)
I20260811 22:16:58.127804 8124 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.130:45941
W20260811 22:16:58.475481 8584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:16:58.475819 8584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:16:58.475972 8584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:16:58.483599 8584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:16:58.483700 8584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:16:58.483862 8584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:16:58.492892 8584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:38487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:16:58.494202 8584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:16:58.496162 8584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:16:58.509971 8592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:58.509918 8593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:16:58.509995 8596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:16:58.510680 8584 server_base.cc:1034] running on GCE node
I20260811 22:16:58.511397 8584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:16:58.512396 8584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:16:58.513676 8584 hybrid_clock.cc:648] HybridClock initialized: now 1786486618513622 us; error 58 us; skew 500 ppm
I20260811 22:16:58.514122 8584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:16:58.516826 8584 webserver.cc:460] Webserver started at http://127.10.173.131:40757/ using document root <none> and password file <none>
I20260811 22:16:58.517625 8584 fs_manager.cc:359] Metadata directory not provided
I20260811 22:16:58.517779 8584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:16:58.518143 8584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:16:58.521422 8584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "76fb9cad74bf4870b37fc1a11a3ff75b"
format_stamp: "Formatted at 2026-08-11 22:16:58 on dist-test-slave-68xk"
server_key: "4c5c4d08bdc6d08a659e697ccc88f9cb"
server_key_iv: "7bd4b68eda2dfff99d6a4bfebb1cbf1f"
server_key_version: "encryptionkey@0"
I20260811 22:16:58.522310 8584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "76fb9cad74bf4870b37fc1a11a3ff75b"
format_stamp: "Formatted at 2026-08-11 22:16:58 on dist-test-slave-68xk"
server_key: "4c5c4d08bdc6d08a659e697ccc88f9cb"
server_key_iv: "7bd4b68eda2dfff99d6a4bfebb1cbf1f"
server_key_version: "encryptionkey@0"
I20260811 22:16:58.527850 8584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 22:16:58.531708 8603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:58.533115 8584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:16:58.533326 8584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "76fb9cad74bf4870b37fc1a11a3ff75b"
format_stamp: "Formatted at 2026-08-11 22:16:58 on dist-test-slave-68xk"
server_key: "4c5c4d08bdc6d08a659e697ccc88f9cb"
server_key_iv: "7bd4b68eda2dfff99d6a4bfebb1cbf1f"
server_key_version: "encryptionkey@0"
I20260811 22:16:58.533562 8584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:16:58.560405 8584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:16:58.561378 8584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:16:58.561753 8584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:16:58.562925 8584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:16:58.564625 8584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:16:58.564760 8584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:58.564910 8584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:16:58.565011 8584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:16:58.603611 8584 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:36785
I20260811 22:16:58.603713 8775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:36785 every 8 connection(s)
I20260811 22:16:58.605082 8584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:16:58.606736 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 8584
I20260811 22:16:58.606946 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:16:58.607518 10934 external_mini_cluster.cc:1428] Setting key 6676672297ecfaa04fb44356e6a2d3e1
I20260811 22:16:58.621953 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.608100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:36117 (local address 127.10.173.190:38487)
0811 22:16:58.608384 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:58.608392 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:58.612656 (+ 4264us) server_negotiation.cc:407] Connection header received
0811 22:16:58.614818 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:58.614825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:58.614921 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:58.615064 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:58.616258 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.617103 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.618877 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.619342 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.619495 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.621019 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:58.621026 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:58.621033 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:58.621061 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:58.621096 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:58.621110 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:58.621220 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:58.621465 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 22:16:58.621581 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260811 22:16:58.623332 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.608820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:38487 (local address 127.10.173.131:36117)
0811 22:16:58.612474 (+ 3654us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:58.612534 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:58.614535 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:58.615336 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:58.615346 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:58.615447 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:16:58.616069 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.616088 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.617294 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.617299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.618722 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.618735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.619594 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.619601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.619866 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.620668 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:16:58.620671 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:16:58.620712 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:16:58.620799 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:16:58.621222 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:16:58.621238 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:16:58.621249 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 22:16:58.621452 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:16:58.621919 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3074,"thread_start_us":2831,"threads_started":1}
I20260811 22:16:58.624912 8777 heartbeater.cc:347] Connected to a master server at 127.10.173.190:38487
I20260811 22:16:58.625268 8777 heartbeater.cc:464] Registering TS with master...
I20260811 22:16:58.626037 8777 heartbeater.cc:511] Master 127.10.173.190:38487 requested a full tablet report, sending...
I20260811 22:16:58.627660 8124 ts_manager.cc:194] Registered new tserver with Master: 76fb9cad74bf4870b37fc1a11a3ff75b (127.10.173.131:36785)
I20260811 22:16:58.628472 8124 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.131:36117
I20260811 22:16:58.634728 8380 heartbeater.cc:503] Master 127.10.173.190:38487 was elected leader, sending a full tablet report...
I20260811 22:16:58.637223 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:16:58.648233 8382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.641920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58694 (local address 127.10.173.190:38487)
0811 22:16:58.642190 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:58.642198 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:58.642557 (+ 359us) server_negotiation.cc:407] Connection header received
0811 22:16:58.642769 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:58.642773 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:58.642836 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:58.642928 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:58.643458 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.644185 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.645587 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.646020 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.646126 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.647437 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:58.647442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:58.647445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:58.647463 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:58.647496 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:58.647508 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:58.647623 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:58.647803 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 22:16:58.647885 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:16:58.652529 8124 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:16:58.659658 8124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:16:58.702291 8799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.673035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49438 (local address 127.10.173.131:36785)
0811 22:16:58.692886 (+ 19851us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:58.692898 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:58.692947 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:16:58.693082 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:58.693094 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:58.693444 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:58.693717 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:58.695040 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.696733 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.699511 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.701011 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.701217 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.701463 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:58.701746 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 22:16:58.701915 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19615,"thread_start_us":19413,"threads_started":1}
I20260811 22:16:58.702266 8800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.672684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:36785 (local address 127.0.0.1:49438)
0811 22:16:58.685011 (+ 12327us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:58.685111 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:58.685399 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:58.693736 (+ 8337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:58.693744 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:58.693785 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:58.694345 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.694359 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.696878 (+ 2519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.696883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.699266 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.699280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.701230 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.701236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.701449 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.701498 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 22:16:58.701628 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 22:16:58.701847 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11860,"mutex_wait_us":80,"spinlock_wait_cycles":7296,"thread_start_us":20739,"threads_started":1}
I20260811 22:16:58.702720 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.673448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54126 (local address 127.10.173.130:37681)
0811 22:16:58.693163 (+ 19715us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:58.693175 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:58.693861 (+ 686us) server_negotiation.cc:407] Connection header received
0811 22:16:58.694212 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:58.694220 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:58.694494 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:58.694675 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:58.695626 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.697048 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.699785 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.701148 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.701307 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.701467 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:58.702146 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 22:16:58.702363 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19528,"thread_start_us":19406,"threads_started":1}
I20260811 22:16:58.702936 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.672883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:37681 (local address 127.0.0.1:54126)
0811 22:16:58.693713 (+ 20830us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:58.693735 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:58.693990 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:58.695012 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:58.695019 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:58.695049 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:58.695452 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.695464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.697198 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.697203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.699562 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.699578 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.701624 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.701633 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.701823 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.701905 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 22:16:58.701994 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:16:58.702533 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20705,"mutex_wait_us":428,"spinlock_wait_cycles":17664,"thread_start_us":21791,"threads_started":1}
I20260811 22:16:58.703737 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.673867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47678 (local address 127.10.173.129:45553)
0811 22:16:58.692234 (+ 18367us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:58.692248 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:58.695237 (+ 2989us) server_negotiation.cc:407] Connection header received
0811 22:16:58.695371 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:58.695380 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:58.695605 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:58.695819 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:58.696368 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.697750 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.700237 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.701222 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.701415 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.701620 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:58.702363 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 22:16:58.703398 (+ 1035us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18116,"thread_start_us":17954,"threads_started":1}
I20260811 22:16:58.704356 8798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.673335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45553 (local address 127.0.0.1:47678)
0811 22:16:58.694973 (+ 21638us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:58.694990 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:58.695094 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:58.695799 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:58.695803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:58.695828 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:58.696198 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.696208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.697913 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.697924 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.700037 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.700047 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.701664 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.701671 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.701922 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.701959 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:16:58.702032 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:16:58.702158 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21502,"mutex_wait_us":25,"spinlock_wait_cycles":384,"thread_start_us":12360,"threads_started":1}
I20260811 22:16:58.706915 8666 tablet_service.cc:1467] Processing CreateTablet for tablet 00c31bf1f15c41c19552c1ef1783d760 (DEFAULT_TABLE table=test-table [id=84f3251219924b64af6e6d7e4667df57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:58.706997 8473 tablet_service.cc:1467] Processing CreateTablet for tablet 00c31bf1f15c41c19552c1ef1783d760 (DEFAULT_TABLE table=test-table [id=84f3251219924b64af6e6d7e4667df57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:58.708271 8666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00c31bf1f15c41c19552c1ef1783d760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:58.708395 8473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00c31bf1f15c41c19552c1ef1783d760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:58.708364 8285 tablet_service.cc:1467] Processing CreateTablet for tablet 00c31bf1f15c41c19552c1ef1783d760 (DEFAULT_TABLE table=test-table [id=84f3251219924b64af6e6d7e4667df57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:16:58.709725 8285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00c31bf1f15c41c19552c1ef1783d760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:16:58.720301 8807 tablet_bootstrap.cc:492] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: Bootstrap starting.
I20260811 22:16:58.722879 8808 tablet_bootstrap.cc:492] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: Bootstrap starting.
I20260811 22:16:58.724203 8807 tablet_bootstrap.cc:654] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:58.725592 8807 log.cc:824] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:58.726709 8808 tablet_bootstrap.cc:654] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:58.728221 8808 log.cc:824] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:58.729197 8809 tablet_bootstrap.cc:492] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: Bootstrap starting.
I20260811 22:16:58.732504 8807 tablet_bootstrap.cc:492] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: No bootstrap required, opened a new log
I20260811 22:16:58.732684 8808 tablet_bootstrap.cc:492] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: No bootstrap required, opened a new log
I20260811 22:16:58.732860 8807 ts_tablet_manager.cc:1397] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260811 22:16:58.732951 8809 tablet_bootstrap.cc:654] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: Neither blocks nor log segments found. Creating new log.
I20260811 22:16:58.732972 8808 ts_tablet_manager.cc:1397] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 22:16:58.734215 8809 log.cc:824] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: Log is configured to *not* fsync() on all Append() calls
I20260811 22:16:58.737416 8809 tablet_bootstrap.cc:492] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: No bootstrap required, opened a new log
I20260811 22:16:58.737684 8809 ts_tablet_manager.cc:1397] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 22:16:58.741734 8807 raft_consensus.cc:348] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 } } peers { permanent_uuid: "0776d30db1374274a82887049ea10d01" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 37681 } } peers { permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 } }
I20260811 22:16:58.742141 8807 raft_consensus.cc:374] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:58.742259 8807 raft_consensus.cc:729] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76fb9cad74bf4870b37fc1a11a3ff75b, State: Initialized, Role: FOLLOWER
I20260811 22:16:58.742954 8807 consensus_queue.cc:260] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 } } peers { permanent_uuid: "0776d30db1374274a82887049ea10d01" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 37681 } } peers { permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 } }
I20260811 22:16:58.743491 8808 raft_consensus.cc:348] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 } } peers { permanent_uuid: "0776d30db1374274a82887049ea10d01" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 37681 } } peers { permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 } }
I20260811 22:16:58.743932 8808 raft_consensus.cc:374] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:58.744035 8808 raft_consensus.cc:729] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0776d30db1374274a82887049ea10d01, State: Initialized, Role: FOLLOWER
I20260811 22:16:58.744196 8777 heartbeater.cc:503] Master 127.10.173.190:38487 was elected leader, sending a full tablet report...
I20260811 22:16:58.744618 8808 consensus_queue.cc:260] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 } } peers { permanent_uuid: "0776d30db1374274a82887049ea10d01" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 37681 } } peers { permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 } }
I20260811 22:16:58.745745 8579 heartbeater.cc:503] Master 127.10.173.190:38487 was elected leader, sending a full tablet report...
I20260811 22:16:58.747018 8807 ts_tablet_manager.cc:1428] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260811 22:16:58.748313 8808 ts_tablet_manager.cc:1428] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: Time spent starting tablet: real 0.015s user 0.000s sys 0.014s
I20260811 22:16:58.748392 8809 raft_consensus.cc:348] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 } } peers { permanent_uuid: "0776d30db1374274a82887049ea10d01" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 37681 } } peers { permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 } }
I20260811 22:16:58.748894 8809 raft_consensus.cc:374] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:16:58.749008 8809 raft_consensus.cc:729] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eeee4de17084e52935dde50712b6034, State: Initialized, Role: FOLLOWER
I20260811 22:16:58.749702 8809 consensus_queue.cc:260] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 } } peers { permanent_uuid: "0776d30db1374274a82887049ea10d01" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 37681 } } peers { permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 } }
I20260811 22:16:58.751843 8809 ts_tablet_manager.cc:1428] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260811 22:16:58.830930 8817 raft_consensus.cc:482] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:16:58.831315 8817 raft_consensus.cc:504] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 } } peers { permanent_uuid: "0776d30db1374274a82887049ea10d01" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 37681 } } peers { permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 } }
I20260811 22:16:58.833139 8817 leader_election.cc:288] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76fb9cad74bf4870b37fc1a11a3ff75b (127.10.173.131:36785), 1eeee4de17084e52935dde50712b6034 (127.10.173.129:45553)
I20260811 22:16:58.843585 8799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.833620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:37633 (local address 127.10.173.131:36785)
0811 22:16:58.833845 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:58.833852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:58.834695 (+ 843us) server_negotiation.cc:407] Connection header received
0811 22:16:58.834963 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:58.834968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:58.835052 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:58.835167 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:58.836306 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.837465 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.841155 (+ 3690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.842469 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.842683 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.842751 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:58.843180 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 22:16:58.843337 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:16:58.843791 8821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.833638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:45553 (local address 127.10.173.130:45679)
0811 22:16:58.834702 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:58.834718 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:58.834824 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:58.835241 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:58.835247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:58.835285 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:58.836034 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.836056 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.837661 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.837669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.840878 (+ 3209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.840888 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.842889 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.842895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.843164 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.843200 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:16:58.843296 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:16:58.843414 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":398,"thread_start_us":383,"threads_started":1}
I20260811 22:16:58.843814 8822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.833528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:36785 (local address 127.10.173.130:37633)
0811 22:16:58.834554 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0811 22:16:58.834575 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:16:58.834763 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:16:58.835241 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:16:58.835247 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:16:58.835369 (+ 122us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:16:58.836127 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.836141 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.837681 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.837686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.840864 (+ 3178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:16:58.840883 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.842698 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.842709 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:16:58.842932 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.842963 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:16:58.843129 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 22:16:58.843301 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":219,"spinlock_wait_cycles":28416,"thread_start_us":775,"threads_started":1}
I20260811 22:16:58.844372 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.833784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:45679 (local address 127.10.173.129:45553)
0811 22:16:58.834004 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:58.834011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:58.834868 (+ 857us) server_negotiation.cc:407] Connection header received
0811 22:16:58.834957 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:58.834961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:58.835062 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:58.835218 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:16:58.836306 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.837475 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.841152 (+ 3677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.842469 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.842645 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.842721 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:58.843357 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 22:16:58.843506 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:16:58.844982 8698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c31bf1f15c41c19552c1ef1783d760" candidate_uuid: "0776d30db1374274a82887049ea10d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" is_pre_election: true
I20260811 22:16:58.845408 8312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c31bf1f15c41c19552c1ef1783d760" candidate_uuid: "0776d30db1374274a82887049ea10d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eeee4de17084e52935dde50712b6034" is_pre_election: true
I20260811 22:16:58.845633 8698 raft_consensus.cc:2454] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0776d30db1374274a82887049ea10d01 in term 0.
I20260811 22:16:58.845942 8312 raft_consensus.cc:2454] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0776d30db1374274a82887049ea10d01 in term 0.
I20260811 22:16:58.846478 8409 leader_election.cc:302] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0776d30db1374274a82887049ea10d01, 76fb9cad74bf4870b37fc1a11a3ff75b; no voters:
I20260811 22:16:58.846906 8817 raft_consensus.cc:2781] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:16:58.847084 8817 raft_consensus.cc:482] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:16:58.847199 8817 raft_consensus.cc:3037] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:58.849587 8817 raft_consensus.cc:504] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 } } peers { permanent_uuid: "0776d30db1374274a82887049ea10d01" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 37681 } } peers { permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 } }
I20260811 22:16:58.850183 8817 leader_election.cc:288] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [CANDIDATE]: Term 1 election: Requested vote from peers 76fb9cad74bf4870b37fc1a11a3ff75b (127.10.173.131:36785), 1eeee4de17084e52935dde50712b6034 (127.10.173.129:45553)
I20260811 22:16:58.850694 8312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c31bf1f15c41c19552c1ef1783d760" candidate_uuid: "0776d30db1374274a82887049ea10d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1eeee4de17084e52935dde50712b6034"
I20260811 22:16:58.850904 8312 raft_consensus.cc:3037] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:58.853125 8312 raft_consensus.cc:2454] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0776d30db1374274a82887049ea10d01 in term 1.
I20260811 22:16:58.853469 8698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c31bf1f15c41c19552c1ef1783d760" candidate_uuid: "0776d30db1374274a82887049ea10d01" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b"
I20260811 22:16:58.853616 8404 leader_election.cc:302] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0776d30db1374274a82887049ea10d01, 1eeee4de17084e52935dde50712b6034; no voters:
I20260811 22:16:58.853688 8698 raft_consensus.cc:3037] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:16:58.853999 8817 raft_consensus.cc:2781] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:16:58.854568 8817 raft_consensus.cc:686] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 1 LEADER]: Becoming Leader. State: Replica: 0776d30db1374274a82887049ea10d01, State: Running, Role: LEADER
I20260811 22:16:58.855155 8817 consensus_queue.cc:237] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 } } peers { permanent_uuid: "0776d30db1374274a82887049ea10d01" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 37681 } } peers { permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 } }
W20260811 22:16:58.856912 8778 tablet.cc:2369] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:16:58.857054 8698 raft_consensus.cc:2454] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0776d30db1374274a82887049ea10d01 in term 1.
W20260811 22:16:58.859481 8580 tablet.cc:2369] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:16:58.861086 8123 catalog_manager.cc:5495] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0776d30db1374274a82887049ea10d01 (127.10.173.130). New cstate: current_term: 1 leader_uuid: "0776d30db1374274a82887049ea10d01" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0776d30db1374274a82887049ea10d01" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 37681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 } health_report { overall_health: UNKNOWN } } }
W20260811 22:16:58.863276 8381 tablet.cc:2369] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:16:58.893452 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.887906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54140 (local address 127.10.173.130:37681)
0811 22:16:58.888215 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:58.888220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:58.888240 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:16:58.888401 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:58.888405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:58.888506 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:58.888640 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:16:58.889015 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.889761 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.891184 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.891605 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.891715 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.892252 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:16:58.892757 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:16:58.892888 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:58.893035 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 22:16:58.893103 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 22:16:58.902144 8312 raft_consensus.cc:1261] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034 [term 1 FOLLOWER]: Refusing update from remote peer 0776d30db1374274a82887049ea10d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:16:58.902187 8698 raft_consensus.cc:1261] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b [term 1 FOLLOWER]: Refusing update from remote peer 0776d30db1374274a82887049ea10d01: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:16:58.903213 8823 consensus_queue.cc:1035] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1eeee4de17084e52935dde50712b6034" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 45553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:16:58.903537 8817 consensus_queue.cc:1035] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76fb9cad74bf4870b37fc1a11a3ff75b" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 36785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:16:58.915365 8828 mvcc.cc:205] Tried to move back new op lower bound from 7317449191013224448 to 7317449190835793920. Current Snapshot: MvccSnapshot[applied={T|T < 7317449191013224448}]
I20260811 22:16:58.929026 8124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:16:58.929356 8124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:16:58.933494 8124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb9b3f717e0b4bb29609ac35cd29c7ba: Sending DeleteTablet for 3 replicas of tablet 00c31bf1f15c41c19552c1ef1783d760
I20260811 22:16:58.934839 8666 tablet_service.cc:1514] Processing DeleteTablet for tablet 00c31bf1f15c41c19552c1ef1783d760 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:58 UTC) from {username='slave'} at 127.0.0.1:49438
I20260811 22:16:58.935036 8285 tablet_service.cc:1514] Processing DeleteTablet for tablet 00c31bf1f15c41c19552c1ef1783d760 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:58 UTC) from {username='slave'} at 127.0.0.1:47678
I20260811 22:16:58.935189 8473 tablet_service.cc:1514] Processing DeleteTablet for tablet 00c31bf1f15c41c19552c1ef1783d760 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:16:58 UTC) from {username='slave'} at 127.0.0.1:54126
I20260811 22:16:58.937068 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 8210
I20260811 22:16:58.948215 8844 tablet_replica.cc:330] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: stopping tablet replica
I20260811 22:16:58.949240 8842 tablet_replica.cc:330] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: stopping tablet replica
I20260811 22:16:58.949733 8842 raft_consensus.cc:2229] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:16:58.950181 8844 raft_consensus.cc:2229] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:16:58.950502 8842 raft_consensus.cc:2258] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:58.951278 8844 raft_consensus.cc:2258] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:58.951398 8841 tablet_replica.cc:330] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: stopping tablet replica
I20260811 22:16:58.951818 8841 raft_consensus.cc:2229] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:16:58.952423 8841 raft_consensus.cc:2258] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:16:58.954125 8844 ts_tablet_manager.cc:1905] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:58.955092 8842 ts_tablet_manager.cc:1905] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:58.957170 8841 ts_tablet_manager.cc:1905] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:16:58.962030 8844 ts_tablet_manager.cc:1918] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:58.962025 8842 ts_tablet_manager.cc:1918] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:58.962203 8842 log.cc:1196] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/00c31bf1f15c41c19552c1ef1783d760
I20260811 22:16:58.962356 8844 log.cc:1196] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/00c31bf1f15c41c19552c1ef1783d760
I20260811 22:16:58.962824 8842 ts_tablet_manager.cc:1939] T 00c31bf1f15c41c19552c1ef1783d760 P 1eeee4de17084e52935dde50712b6034: Deleting consensus metadata
I20260811 22:16:58.963140 8844 ts_tablet_manager.cc:1939] T 00c31bf1f15c41c19552c1ef1783d760 P 0776d30db1374274a82887049ea10d01: Deleting consensus metadata
I20260811 22:16:58.964252 8841 ts_tablet_manager.cc:1918] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:16:58.964416 8841 log.cc:1196] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/00c31bf1f15c41c19552c1ef1783d760
I20260811 22:16:58.964939 8101 catalog_manager.cc:4841] TS 0776d30db1374274a82887049ea10d01 (127.10.173.130:37681): tablet 00c31bf1f15c41c19552c1ef1783d760 (table test-table [id=84f3251219924b64af6e6d7e4667df57]) successfully deleted
I20260811 22:16:58.965180 8841 ts_tablet_manager.cc:1939] T 00c31bf1f15c41c19552c1ef1783d760 P 76fb9cad74bf4870b37fc1a11a3ff75b: Deleting consensus metadata
I20260811 22:16:58.965404 8100 catalog_manager.cc:4841] TS 1eeee4de17084e52935dde50712b6034 (127.10.173.129:45553): tablet 00c31bf1f15c41c19552c1ef1783d760 (table test-table [id=84f3251219924b64af6e6d7e4667df57]) successfully deleted
I20260811 22:16:58.966946 8104 catalog_manager.cc:4841] TS 76fb9cad74bf4870b37fc1a11a3ff75b (127.10.173.131:36785): tablet 00c31bf1f15c41c19552c1ef1783d760 (table test-table [id=84f3251219924b64af6e6d7e4667df57]) successfully deleted
I20260811 22:16:58.967123 8806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:58.938531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47688 (local address 127.10.173.129:45553)
0811 22:16:58.938722 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:58.938729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:58.948424 (+ 9695us) server_negotiation.cc:407] Connection header received
0811 22:16:58.948686 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:58.948697 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:58.948858 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:58.949008 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:58.949810 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.951111 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.953527 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:58.954218 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:58.954321 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:58.965443 (+ 11122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:58.965457 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:58.965474 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:58.966153 (+ 679us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:58.966212 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:58.966234 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:58.966313 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:58.966630 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 22:16:58.966761 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:16:58.967844 8373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:59.112727 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 8210
I20260811 22:16:59.146401 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 8384
I20260811 22:16:59.155486 8801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:59.147290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54152 (local address 127.10.173.130:37681)
0811 22:16:59.147484 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:59.147491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:59.147665 (+ 174us) server_negotiation.cc:407] Connection header received
0811 22:16:59.147833 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:59.147839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:59.147945 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:59.148059 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:59.148556 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:59.149837 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:59.151722 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:59.152456 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:59.152583 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:59.153768 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:59.153779 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:59.153798 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:59.154619 (+ 821us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:59.154678 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:59.154709 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:59.154827 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:59.155006 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 22:16:59.155117 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:16:59.156342 8567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:59.271150 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 8384
I20260811 22:16:59.301309 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 8584
I20260811 22:16:59.308650 8799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:16:59.302156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49448 (local address 127.10.173.131:36785)
0811 22:16:59.302368 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:16:59.302375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:16:59.302514 (+ 139us) server_negotiation.cc:407] Connection header received
0811 22:16:59.302764 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:16:59.302769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:16:59.302852 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:16:59.302961 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:16:59.303398 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:59.304114 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:59.305607 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:16:59.306008 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:16:59.306146 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:16:59.307288 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:16:59.307297 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:16:59.307312 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:16:59.307867 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:16:59.307916 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:16:59.307937 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:16:59.308068 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:16:59.308213 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 22:16:59.308330 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:16:59.309379 8764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:59.399893 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 8584
I20260811 22:16:59.426020 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 8081
I20260811 22:16:59.427229 8168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:16:59.539031 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 8081
2026-08-11T22:16:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3100 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:16:59 dist-test-slave-68xk krb5kdc[8858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:16:59 dist-test-slave-68xk krb5kdc[8858](info): set up 2 sockets
Aug 11 22:16:59 dist-test-slave-68xk krb5kdc[8858](info): commencing operation
krb5kdc: starting...
W20260811 22:17:01.002592 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.409s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:01 dist-test-slave-68xk krb5kdc[8858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:17:01.331037 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:36019
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:36019
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:01.705899 8877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:01.706274 8877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:01.706398 8877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:01.713929 8877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:01.714033 8877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:01.714082 8877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:01.714139 8877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:01.714184 8877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:01.724206 8877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:36019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:36019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:01.725481 8877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:01.726830 8877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:01.739267 8886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:01.739254 8888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:01.739496 8891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:01.740114 8877 server_base.cc:1034] running on GCE node
I20260811 22:17:01.741019 8877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:01.742630 8877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:01.743892 8877 hybrid_clock.cc:648] HybridClock initialized: now 1786486621743830 us; error 52 us; skew 500 ppm
I20260811 22:17:01.744251 8877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:01 dist-test-slave-68xk krb5kdc[8858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486621, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:01.747498 8877 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:17:01.748961 8877 webserver.cc:460] Webserver started at http://127.10.173.190:46179/ using document root <none> and password file <none>
I20260811 22:17:01.749622 8877 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:01.749724 8877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:01.749980 8877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:01.752993 8877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "9d10c18b41474b969afb7549a30eba4d"
format_stamp: "Formatted at 2026-08-11 22:17:01 on dist-test-slave-68xk"
server_key: "e811a023f0309b20a3fb6e7488457003"
server_key_iv: "eeb35f148dfbc1b388cfdf8bae7c0828"
server_key_version: "encryptionkey@0"
I20260811 22:17:01.753775 8877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "9d10c18b41474b969afb7549a30eba4d"
format_stamp: "Formatted at 2026-08-11 22:17:01 on dist-test-slave-68xk"
server_key: "e811a023f0309b20a3fb6e7488457003"
server_key_iv: "eeb35f148dfbc1b388cfdf8bae7c0828"
server_key_version: "encryptionkey@0"
I20260811 22:17:01.759177 8877 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 22:17:01.762746 8899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:01.764182 8877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:17:01.764366 8877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "9d10c18b41474b969afb7549a30eba4d"
format_stamp: "Formatted at 2026-08-11 22:17:01 on dist-test-slave-68xk"
server_key: "e811a023f0309b20a3fb6e7488457003"
server_key_iv: "eeb35f148dfbc1b388cfdf8bae7c0828"
server_key_version: "encryptionkey@0"
I20260811 22:17:01.764578 8877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:01.790696 8877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:01.795408 8877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:01.795732 8877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:01.815686 8877 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:36019
I20260811 22:17:01.815716 8971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:36019 every 8 connection(s)
I20260811 22:17:01.817090 8877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:01.820930 8972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:01.824918 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 8877
I20260811 22:17:01.825148 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:01.825642 10934 external_mini_cluster.cc:1428] Setting key c23b8a09da1ab10a89d1445ea26f5a29
I20260811 22:17:01.831996 8972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d: Bootstrap starting.
I20260811 22:17:01.835877 8972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:01.837080 8972 log.cc:824] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:01.840204 8972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d: No bootstrap required, opened a new log
Aug 11 22:17:01 dist-test-slave-68xk krb5kdc[8858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:17:01.850718 8972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d10c18b41474b969afb7549a30eba4d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36019 } }
I20260811 22:17:01.851222 8977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:01.827460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40736 (local address 127.10.173.190:36019)
0811 22:17:01.828206 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:01.828217 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:01.828257 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:17:01.830380 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:01.830403 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:01.830838 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:01.831149 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:01.833129 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:01.834540 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:01.836513 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:01.837382 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:01.837593 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:01.842653 (+ 5060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:01.842663 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:01.842682 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:01.842732 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:01.846520 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:01.847546 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:01.847553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:01.847565 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:01.847747 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:01.848231 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:01.848235 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:01.848237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:01.848785 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:01.849023 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:01.849556 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 22:17:01.849964 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":343,"threads_started":1}
I20260811 22:17:01.851199 8972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:01.851428 8972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d10c18b41474b969afb7549a30eba4d, State: Initialized, Role: FOLLOWER
I20260811 22:17:01.852142 8972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d10c18b41474b969afb7549a30eba4d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36019 } }
I20260811 22:17:01.852392 8972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:01.852494 8972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:01.852696 8972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:01.856053 8972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d10c18b41474b969afb7549a30eba4d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36019 } }
I20260811 22:17:01.856540 8972 leader_election.cc:302] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d10c18b41474b969afb7549a30eba4d; no voters:
I20260811 22:17:01.857141 8972 leader_election.cc:288] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:01.857304 8980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:01.858278 8980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [term 1 LEADER]: Becoming Leader. State: Replica: 9d10c18b41474b969afb7549a30eba4d, State: Running, Role: LEADER
I20260811 22:17:01.858820 8980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d10c18b41474b969afb7549a30eba4d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36019 } }
I20260811 22:17:01.859632 8972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:01.861979 8982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d10c18b41474b969afb7549a30eba4d. Latest consensus state: current_term: 1 leader_uuid: "9d10c18b41474b969afb7549a30eba4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d10c18b41474b969afb7549a30eba4d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36019 } } }
I20260811 22:17:01.861965 8981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d10c18b41474b969afb7549a30eba4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d10c18b41474b969afb7549a30eba4d" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36019 } } }
I20260811 22:17:01.862458 8982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:01.862458 8981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:01.863348 8988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:01.869614 8988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:01.879448 8988 catalog_manager.cc:1347] Generated new cluster ID: 58aebcaa70ef4df4a05a5084f075d3a4
I20260811 22:17:01.879575 8988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:01.899194 8988 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:01.901134 8988 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:01.909365 8988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d: Generated new TSK 0
I20260811 22:17:01.910459 8988 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.173.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
Entry for principal kudu/127.10.173.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab.
I20260811 22:17:01.996068 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36019
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:02.342855 9014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:02.343143 9014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:02.343302 9014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:02.351130 9014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:02.351239 9014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:02.351442 9014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:17:02.361637 9014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.129.keytab
--principal=kudu/127.10.173.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:02.363039 9014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:02.364377 9014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:02.378687 9019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:02.378751 9022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:02.378773 9020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:02.384395 9014 server_base.cc:1034] running on GCE node
I20260811 22:17:02.385156 9014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:02.386297 9014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:02.387526 9014 hybrid_clock.cc:648] HybridClock initialized: now 1786486622387470 us; error 54 us; skew 500 ppm
I20260811 22:17:02.387904 9014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:02 dist-test-slave-68xk krb5kdc[8858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486622, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:02.391520 9014 init.cc:377] Logged in from keytab as kudu/127.10.173.129@KRBTEST.COM (short username kudu)
I20260811 22:17:02.393181 9014 webserver.cc:460] Webserver started at http://127.10.173.129:42869/ using document root <none> and password file <none>
I20260811 22:17:02.393882 9014 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:02.393985 9014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:02.394260 9014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:02.397251 9014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "8249a50251794775978676eb852393b9"
format_stamp: "Formatted at 2026-08-11 22:17:02 on dist-test-slave-68xk"
server_key: "5d36e0cf5a3e0a35d2ee8e280e24d592"
server_key_iv: "ae3175fe07ba8b7bb1cccc6bc6d7e1ce"
server_key_version: "encryptionkey@0"
I20260811 22:17:02.397996 9014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "8249a50251794775978676eb852393b9"
format_stamp: "Formatted at 2026-08-11 22:17:02 on dist-test-slave-68xk"
server_key: "5d36e0cf5a3e0a35d2ee8e280e24d592"
server_key_iv: "ae3175fe07ba8b7bb1cccc6bc6d7e1ce"
server_key_version: "encryptionkey@0"
I20260811 22:17:02.403363 9014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 22:17:02.406967 9030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:02.408459 9014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:17:02.408675 9014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "8249a50251794775978676eb852393b9"
format_stamp: "Formatted at 2026-08-11 22:17:02 on dist-test-slave-68xk"
server_key: "5d36e0cf5a3e0a35d2ee8e280e24d592"
server_key_iv: "ae3175fe07ba8b7bb1cccc6bc6d7e1ce"
server_key_version: "encryptionkey@0"
I20260811 22:17:02.408908 9014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:02.444796 9014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:02.449278 9014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:02.449596 9014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:02.450695 9014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:17:02.452116 9014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:17:02.452224 9014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:02.452382 9014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:17:02.452459 9014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:02.489012 9014 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:41449
I20260811 22:17:02.489109 9188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:41449 every 8 connection(s)
I20260811 22:17:02.490358 9014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:17:02.498135 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 9014
I20260811 22:17:02.498350 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:17:02.498894 10934 external_mini_cluster.cc:1428] Setting key 771ccae57014201ff8c4a402240effb8
Aug 11 22:17:02 dist-test-slave-68xk krb5kdc[8858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486622, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.129@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:17:02.512326 9194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:02.493338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:59861 (local address 127.10.173.190:36019)
0811 22:17:02.493936 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:02.493941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:02.494856 (+ 915us) server_negotiation.cc:407] Connection header received
0811 22:17:02.497146 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:02.497152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:02.497252 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:02.497383 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:02.499112 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:02.500019 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:02.501796 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:02.502208 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:02.502366 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:02.506267 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:02.506273 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:02.506276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:02.506310 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:02.509092 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:02.509869 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:02.509877 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:02.509886 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:02.510010 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:02.510500 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:02.510510 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:02.510512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:02.510788 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:02.510955 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:02.511727 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 22:17:02.511966 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":360,"threads_started":1}
I20260811 22:17:02.513528 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:02.493821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36019 (local address 127.10.173.129:59861)
0811 22:17:02.494668 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:02.494717 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:02.496837 (+ 2120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:02.497628 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:02.497636 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:02.498154 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:02.498882 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:02.498901 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:02.500184 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:02.500189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:02.501636 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:02.501650 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:02.502497 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:02.502508 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:02.502763 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:02.503492 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:02.503516 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:02.506029 (+ 2513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:02.509257 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:02.509267 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:02.509284 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:02.509731 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:02.510127 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:02.510131 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:02.510135 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:02.510351 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:02.510928 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:02.510937 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:02.511304 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 22:17:02.511594 (+ 290us) client_negotiation.cc:240] Negotiation successful
0811 22:17:02.512144 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":330,"threads_started":1}
I20260811 22:17:02.515362 9189 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36019
I20260811 22:17:02.515735 9189 heartbeater.cc:464] Registering TS with master...
I20260811 22:17:02.516537 9189 heartbeater.cc:511] Master 127.10.173.190:36019 requested a full tablet report, sending...
I20260811 22:17:02.519186 8923 ts_manager.cc:194] Registered new tserver with Master: 8249a50251794775978676eb852393b9 (127.10.173.129:41449)
I20260811 22:17:02.520689 8923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.129@KRBTEST.COM'} at 127.10.173.129:59861
WARNING: no policy specified for kudu/127.10.173.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
Entry for principal kudu/127.10.173.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab.
I20260811 22:17:02.570658 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36019
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:02.927785 9198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:02.928051 9198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:02.928144 9198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:02.935570 9198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:02.935703 9198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:02.935902 9198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:17:02.945689 9198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.130.keytab
--principal=kudu/127.10.173.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:02.947064 9198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:02.948441 9198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:02.962328 9212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:02.962284 9213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:02.962477 9216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:02.963125 9198 server_base.cc:1034] running on GCE node
I20260811 22:17:02.963864 9198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:02.965113 9198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:02.966365 9198 hybrid_clock.cc:648] HybridClock initialized: now 1786486622966312 us; error 55 us; skew 500 ppm
I20260811 22:17:02.966751 9198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:02 dist-test-slave-68xk krb5kdc[8858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486622, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:02.970639 9198 init.cc:377] Logged in from keytab as kudu/127.10.173.130@KRBTEST.COM (short username kudu)
I20260811 22:17:02.972285 9198 webserver.cc:460] Webserver started at http://127.10.173.130:36245/ using document root <none> and password file <none>
I20260811 22:17:02.973079 9198 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:02.973230 9198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:02.973613 9198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:02.976871 9198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "6b6c7ef1e6034a8580f4d14de59ae24d"
format_stamp: "Formatted at 2026-08-11 22:17:02 on dist-test-slave-68xk"
server_key: "06fd9d62d78144b1a0b328692ff3be3c"
server_key_iv: "a4c1ad4c524557124bc572fa84de746a"
server_key_version: "encryptionkey@0"
I20260811 22:17:02.977730 9198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "6b6c7ef1e6034a8580f4d14de59ae24d"
format_stamp: "Formatted at 2026-08-11 22:17:02 on dist-test-slave-68xk"
server_key: "06fd9d62d78144b1a0b328692ff3be3c"
server_key_iv: "a4c1ad4c524557124bc572fa84de746a"
server_key_version: "encryptionkey@0"
I20260811 22:17:02.983377 9198 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 22:17:02.987121 9225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:02.988535 9198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:17:02.988731 9198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "6b6c7ef1e6034a8580f4d14de59ae24d"
format_stamp: "Formatted at 2026-08-11 22:17:02 on dist-test-slave-68xk"
server_key: "06fd9d62d78144b1a0b328692ff3be3c"
server_key_iv: "a4c1ad4c524557124bc572fa84de746a"
server_key_version: "encryptionkey@0"
I20260811 22:17:02.988940 9198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:03.013023 9198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:03.019065 9198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:03.019415 9198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:03.020548 9198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:17:03.022137 9198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:17:03.022226 9198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:03.022372 9198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:17:03.022509 9198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 22:17:03.057331 9198 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:45645
I20260811 22:17:03.057369 9385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:45645 every 8 connection(s)
I20260811 22:17:03.058744 9198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:17:03.064512 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 9198
I20260811 22:17:03.064723 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:17:03.065260 10934 external_mini_cluster.cc:1428] Setting key 2cd7b748fdab6e9b8a99024305d99416
Aug 11 22:17:03 dist-test-slave-68xk krb5kdc[8858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486622, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.130@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:17:03.095139 9389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.061549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:48069 (local address 127.10.173.190:36019)
0811 22:17:03.062021 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:03.062025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:03.063069 (+ 1044us) server_negotiation.cc:407] Connection header received
0811 22:17:03.065058 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:03.065065 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:03.065162 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:03.065288 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:03.067214 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.068429 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.070206 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.071084 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.071242 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.088092 (+ 16850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:03.088098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:03.088102 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:03.088143 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:03.091268 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:03.092151 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:03.092156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:03.092159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:03.092329 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:03.092944 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:03.092948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:03.092951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:03.093356 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:03.093527 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:03.094401 (+ 874us) server_negotiation.cc:299] Negotiation successful
0811 22:17:03.094696 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":233,"threads_started":1}
I20260811 22:17:03.096669 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.062150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36019 (local address 127.10.173.130:48069)
0811 22:17:03.062901 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:03.062947 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:03.064738 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:03.065529 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:03.065537 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:03.066213 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:03.067017 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.067042 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.068674 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.068679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.070036 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.070049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.071311 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.071315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.071513 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.084929 (+ 13416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:03.084960 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:03.087771 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:03.091445 (+ 3674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:03.091457 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:03.091478 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:03.091971 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:03.092460 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:03.092466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:03.092470 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:03.092781 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:03.093514 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:03.093524 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:03.093979 (+ 455us) client_negotiation.cc:769] Sending connection context
0811 22:17:03.094295 (+ 316us) client_negotiation.cc:240] Negotiation successful
0811 22:17:03.094926 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":336,"threads_started":1}
I20260811 22:17:03.098518 9387 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36019
I20260811 22:17:03.098857 9387 heartbeater.cc:464] Registering TS with master...
I20260811 22:17:03.099639 9387 heartbeater.cc:511] Master 127.10.173.190:36019 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.173.131@KRBTEST.COM; defaulting to no policy
I20260811 22:17:03.101296 8923 ts_manager.cc:194] Registered new tserver with Master: 6b6c7ef1e6034a8580f4d14de59ae24d (127.10.173.130:45645)
I20260811 22:17:03.102267 8923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.130@KRBTEST.COM'} at 127.10.173.130:48069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
Entry for principal kudu/127.10.173.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab.
I20260811 22:17:03.139506 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36019
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:37963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:03.483841 9394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:03.484144 9394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:03.484320 9394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:03.491847 9394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:03.492015 9394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:03.492265 9394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:17:03.502842 9394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.131.keytab
--principal=kudu/127.10.173.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:36019
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:03.504185 9394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:03.505609 9394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:03.519868 9401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:03.519868 9405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:03.519868 9402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:03.521270 9394 server_base.cc:1034] running on GCE node
I20260811 22:17:03.521915 9394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:03.522959 9394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:03.523852 9189 heartbeater.cc:503] Master 127.10.173.190:36019 was elected leader, sending a full tablet report...
I20260811 22:17:03.524223 9394 hybrid_clock.cc:648] HybridClock initialized: now 1786486623524174 us; error 49 us; skew 500 ppm
I20260811 22:17:03.524552 9394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:03 dist-test-slave-68xk krb5kdc[8858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:03.527953 9394 init.cc:377] Logged in from keytab as kudu/127.10.173.131@KRBTEST.COM (short username kudu)
I20260811 22:17:03.529517 9394 webserver.cc:460] Webserver started at http://127.10.173.131:37465/ using document root <none> and password file <none>
I20260811 22:17:03.530217 9394 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:03.530370 9394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:03.530748 9394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:03.533656 9394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "5532ce3adc024dd98dd648e9b2f2a749"
format_stamp: "Formatted at 2026-08-11 22:17:03 on dist-test-slave-68xk"
server_key: "f36ee4aecf6c103504fef1a083872151"
server_key_iv: "f97bdb7cdb8d030f579ef8f7e05497e0"
server_key_version: "encryptionkey@0"
I20260811 22:17:03.534531 9394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "5532ce3adc024dd98dd648e9b2f2a749"
format_stamp: "Formatted at 2026-08-11 22:17:03 on dist-test-slave-68xk"
server_key: "f36ee4aecf6c103504fef1a083872151"
server_key_iv: "f97bdb7cdb8d030f579ef8f7e05497e0"
server_key_version: "encryptionkey@0"
I20260811 22:17:03.539790 9394 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:17:03.543394 9416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:03.544731 9394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:17:03.544916 9394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "5532ce3adc024dd98dd648e9b2f2a749"
format_stamp: "Formatted at 2026-08-11 22:17:03 on dist-test-slave-68xk"
server_key: "f36ee4aecf6c103504fef1a083872151"
server_key_iv: "f97bdb7cdb8d030f579ef8f7e05497e0"
server_key_version: "encryptionkey@0"
I20260811 22:17:03.545140 9394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:03.579447 9394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:03.584079 9394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:03.584411 9394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:03.585466 9394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:17:03.587065 9394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:17:03.587182 9394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:03.587347 9394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:17:03.587432 9394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:03.627516 9394 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:39251
I20260811 22:17:03.627558 9581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:39251 every 8 connection(s)
I20260811 22:17:03.629159 9394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:17:03.632012 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 9394
I20260811 22:17:03.632210 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:17:03.632670 10934 external_mini_cluster.cc:1428] Setting key d944ce84e5463a1f2ed4db8aa9ad0b7b
Aug 11 22:17:03 dist-test-slave-68xk krb5kdc[8858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486623, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.131@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:17:03.651858 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.633909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:43745 (local address 127.10.173.190:36019)
0811 22:17:03.634457 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:03.634464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:03.635780 (+ 1316us) server_negotiation.cc:407] Connection header received
0811 22:17:03.637636 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:03.637640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:03.637708 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:03.637798 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:03.639314 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.640055 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.641518 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.641894 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.641993 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.645400 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:03.645406 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:03.645410 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:03.645448 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:03.648938 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:03.649639 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:03.649646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:03.649649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:03.649777 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:03.650248 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:03.650251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:03.650253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:03.650587 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:03.650712 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:03.651416 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 22:17:03.651577 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":256,"threads_started":1}
I20260811 22:17:03.652932 9591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.634711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:36019 (local address 127.10.173.131:43745)
0811 22:17:03.635585 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:03.635660 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:03.637377 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:03.638000 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:03.638008 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:03.638603 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:03.639162 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.639180 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.640194 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.640197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.641390 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.641402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.642088 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.642093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.642287 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.643013 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:03.643061 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:03.645185 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:03.649075 (+ 3890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:03.649084 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:03.649105 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:03.649506 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:03.649910 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:03.649914 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:03.649917 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:03.650135 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:03.650725 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:03.650731 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:03.651063 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 22:17:03.651318 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 22:17:03.651744 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":321,"threads_started":1}
I20260811 22:17:03.654479 9583 heartbeater.cc:347] Connected to a master server at 127.10.173.190:36019
I20260811 22:17:03.654767 9583 heartbeater.cc:464] Registering TS with master...
I20260811 22:17:03.655507 9583 heartbeater.cc:511] Master 127.10.173.190:36019 requested a full tablet report, sending...
I20260811 22:17:03.656894 8923 ts_manager.cc:194] Registered new tserver with Master: 5532ce3adc024dd98dd648e9b2f2a749 (127.10.173.131:39251)
I20260811 22:17:03.657776 8923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.173.131@KRBTEST.COM'} at 127.10.173.131:43745
I20260811 22:17:03.661113 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:17:03.678972 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.665558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.10.173.190:36019)
0811 22:17:03.665813 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:03.665822 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:03.666185 (+ 363us) server_negotiation.cc:407] Connection header received
0811 22:17:03.666302 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:03.666306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:03.666384 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:03.666558 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:03.667927 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.668690 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.670175 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.670693 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.670804 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.673214 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:03.673219 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:03.673223 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:03.673255 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:03.676280 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:03.676965 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:03.676970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:03.676973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:03.677134 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:03.677507 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:03.677511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:03.677513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:03.677850 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:03.678035 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:03.678371 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:17:03.678614 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 22:17:03.683137 8923 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:17:03.690510 8923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:17:03.718042 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.704259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:45645 (local address 127.0.0.1:34878)
0811 22:17:03.705642 (+ 1383us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:03.705681 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:03.705869 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:03.707112 (+ 1243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:03.707121 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:03.707154 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:17:03.707621 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.707640 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.710217 (+ 2577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.710226 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.713511 (+ 3285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.713532 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.716565 (+ 3033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.716576 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.716880 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.716961 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 22:17:03.717157 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:17:03.717436 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1142,"thread_start_us":973,"threads_started":1}
I20260811 22:17:03.719292 9604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.704473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.10.173.130:45645)
0811 22:17:03.705155 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:03.705167 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:03.706100 (+ 933us) server_negotiation.cc:407] Connection header received
0811 22:17:03.706217 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:03.706227 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:03.706539 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:03.706758 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:17:03.707872 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.709981 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.713747 (+ 3766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.715893 (+ 2146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.716131 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.716372 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:03.718705 (+ 2333us) server_negotiation.cc:299] Negotiation successful
0811 22:17:03.718925 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":322,"threads_started":1}
I20260811 22:17:03.723706 9290 tablet_service.cc:1467] Processing CreateTablet for tablet 89dd93ce76cf463680b469e2d55eb37d (DEFAULT_TABLE table=test-table [id=d6e7bfefc7264518b5c36aad6000a8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:17:03.725212 9290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89dd93ce76cf463680b469e2d55eb37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:03.726886 9603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.707232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:41449 (local address 127.0.0.1:35324)
0811 22:17:03.718352 (+ 11120us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:03.718376 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:03.718511 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:03.719827 (+ 1316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:03.719832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:03.719861 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:17:03.720042 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.720052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.721808 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.721814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.724048 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.724066 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.725941 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.725950 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.726165 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.726207 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 22:17:03.726322 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:17:03.726494 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11012,"thread_start_us":10906,"threads_started":1}
I20260811 22:17:03.727933 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.706870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35324 (local address 127.10.173.129:41449)
0811 22:17:03.719091 (+ 12221us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:03.719100 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:03.719136 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:17:03.719248 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:03.719256 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:03.719635 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:03.719837 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:17:03.720207 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.721625 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.724246 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.725753 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.725956 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.726118 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:03.727385 (+ 1267us) server_negotiation.cc:299] Negotiation successful
0811 22:17:03.727584 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12026,"thread_start_us":12030,"threads_started":1}
I20260811 22:17:03.730537 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.704854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:39251 (local address 127.0.0.1:49858)
0811 22:17:03.717403 (+ 12549us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:03.717430 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:03.717587 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:03.722140 (+ 4553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:03.722146 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:03.722174 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:17:03.722675 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.722690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.724780 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.724789 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.726759 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.726776 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.729699 (+ 2923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.729707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.729895 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.729934 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:17:03.730023 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:17:03.730156 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12362,"thread_start_us":13057,"threads_started":1}
I20260811 22:17:03.730723 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.705565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.10.173.131:39251)
0811 22:17:03.721308 (+ 15743us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:03.721315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:03.721347 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:17:03.721429 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:03.721435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:03.721598 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:03.721826 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:17:03.722940 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.724637 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.727001 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.729469 (+ 2468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.729716 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.729912 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:03.730133 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 22:17:03.730308 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15562,"thread_start_us":15384,"threads_started":1}
I20260811 22:17:03.732191 9099 tablet_service.cc:1467] Processing CreateTablet for tablet 89dd93ce76cf463680b469e2d55eb37d (DEFAULT_TABLE table=test-table [id=d6e7bfefc7264518b5c36aad6000a8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:17:03.733915 9099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89dd93ce76cf463680b469e2d55eb37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:03.734874 9474 tablet_service.cc:1467] Processing CreateTablet for tablet 89dd93ce76cf463680b469e2d55eb37d (DEFAULT_TABLE table=test-table [id=d6e7bfefc7264518b5c36aad6000a8f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:17:03.736491 9474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89dd93ce76cf463680b469e2d55eb37d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:03.739254 9612 tablet_bootstrap.cc:492] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: Bootstrap starting.
I20260811 22:17:03.742282 9612 tablet_bootstrap.cc:654] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:03.743418 9612 log.cc:824] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:03.746480 9612 tablet_bootstrap.cc:492] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: No bootstrap required, opened a new log
I20260811 22:17:03.746778 9612 ts_tablet_manager.cc:1397] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260811 22:17:03.751058 9614 tablet_bootstrap.cc:492] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: Bootstrap starting.
I20260811 22:17:03.752921 9617 tablet_bootstrap.cc:492] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: Bootstrap starting.
I20260811 22:17:03.756062 9614 tablet_bootstrap.cc:654] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:03.755885 9612 raft_consensus.cc:348] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45645 } } peers { permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 } } peers { permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 } }
I20260811 22:17:03.756270 9617 tablet_bootstrap.cc:654] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:03.756340 9612 raft_consensus.cc:374] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:03.756458 9612 raft_consensus.cc:729] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b6c7ef1e6034a8580f4d14de59ae24d, State: Initialized, Role: FOLLOWER
I20260811 22:17:03.757139 9612 consensus_queue.cc:260] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45645 } } peers { permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 } } peers { permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 } }
I20260811 22:17:03.757637 9617 log.cc:824] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:03.757817 9614 log.cc:824] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:03.758500 9387 heartbeater.cc:503] Master 127.10.173.190:36019 was elected leader, sending a full tablet report...
I20260811 22:17:03.759261 9612 ts_tablet_manager.cc:1428] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
I20260811 22:17:03.761951 9617 tablet_bootstrap.cc:492] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: No bootstrap required, opened a new log
I20260811 22:17:03.762281 9617 ts_tablet_manager.cc:1397] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260811 22:17:03.762358 9614 tablet_bootstrap.cc:492] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: No bootstrap required, opened a new log
I20260811 22:17:03.762722 9614 ts_tablet_manager.cc:1397] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260811 22:17:03.770738 9617 raft_consensus.cc:348] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45645 } } peers { permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 } } peers { permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 } }
I20260811 22:17:03.771128 9617 raft_consensus.cc:374] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:03.771248 9617 raft_consensus.cc:729] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5532ce3adc024dd98dd648e9b2f2a749, State: Initialized, Role: FOLLOWER
I20260811 22:17:03.771587 9614 raft_consensus.cc:348] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45645 } } peers { permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 } } peers { permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 } }
I20260811 22:17:03.771946 9614 raft_consensus.cc:374] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:03.771881 9617 consensus_queue.cc:260] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45645 } } peers { permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 } } peers { permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 } }
I20260811 22:17:03.772080 9614 raft_consensus.cc:729] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8249a50251794775978676eb852393b9, State: Initialized, Role: FOLLOWER
I20260811 22:17:03.772770 9614 consensus_queue.cc:260] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45645 } } peers { permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 } } peers { permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 } }
I20260811 22:17:03.773154 9583 heartbeater.cc:503] Master 127.10.173.190:36019 was elected leader, sending a full tablet report...
I20260811 22:17:03.773913 9617 ts_tablet_manager.cc:1428] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 22:17:03.774694 9614 ts_tablet_manager.cc:1428] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
W20260811 22:17:03.811066 9388 tablet.cc:2369] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:17:03.881338 9585 tablet.cc:2369] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:17:03.922955 9618 raft_consensus.cc:482] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:17:03.923359 9618 raft_consensus.cc:504] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45645 } } peers { permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 } } peers { permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 } }
I20260811 22:17:03.925307 9618 leader_election.cc:288] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8249a50251794775978676eb852393b9 (127.10.173.129:41449), 5532ce3adc024dd98dd648e9b2f2a749 (127.10.173.131:39251)
I20260811 22:17:03.933176 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.925771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:33237 (local address 127.10.173.131:39251)
0811 22:17:03.926111 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:03.926120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:03.927036 (+ 916us) server_negotiation.cc:407] Connection header received
0811 22:17:03.927334 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:03.927339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:03.927406 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:03.927516 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:17:03.928169 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.928968 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.931230 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.932104 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.932305 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.932401 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:03.932730 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 22:17:03.932877 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 22:17:03.933249 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.925767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:47487 (local address 127.10.173.129:41449)
0811 22:17:03.926094 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:03.926112 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:03.927453 (+ 1341us) server_negotiation.cc:407] Connection header received
0811 22:17:03.927695 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:03.927700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:03.927766 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:03.927877 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:17:03.928529 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.929324 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.931349 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.932204 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.932411 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.932496 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:03.932833 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 22:17:03.932976 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:17:03.933337 9624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.925766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:39251 (local address 127.10.173.130:33237)
0811 22:17:03.926907 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:03.926924 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:03.927155 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:03.927538 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:03.927543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:03.927584 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:17:03.927989 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.927999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.929159 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.929163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.931046 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.931067 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.932364 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.932371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.932541 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.932571 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:17:03.932699 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 22:17:03.932845 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"mutex_wait_us":24,"thread_start_us":697,"threads_started":1}
I20260811 22:17:03.933882 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:03.926324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:41449 (local address 127.10.173.130:47487)
0811 22:17:03.927330 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:03.927345 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:03.927517 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:03.927908 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:03.927913 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:03.927954 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:17:03.928362 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.928372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.929500 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.929506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.931179 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:03.931192 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:03.932455 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:03.932462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:03.932658 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:03.932684 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:17:03.932802 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:17:03.932941 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":1,"thread_start_us":603,"threads_started":1}
I20260811 22:17:03.934718 9520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dd93ce76cf463680b469e2d55eb37d" candidate_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5532ce3adc024dd98dd648e9b2f2a749" is_pre_election: true
I20260811 22:17:03.934914 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dd93ce76cf463680b469e2d55eb37d" candidate_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8249a50251794775978676eb852393b9" is_pre_election: true
I20260811 22:17:03.935272 9520 raft_consensus.cc:2454] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b6c7ef1e6034a8580f4d14de59ae24d in term 0.
I20260811 22:17:03.935617 9125 raft_consensus.cc:2454] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b6c7ef1e6034a8580f4d14de59ae24d in term 0.
I20260811 22:17:03.936149 9230 leader_election.cc:302] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5532ce3adc024dd98dd648e9b2f2a749, 6b6c7ef1e6034a8580f4d14de59ae24d; no voters:
I20260811 22:17:03.936653 9618 raft_consensus.cc:2781] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:17:03.936831 9618 raft_consensus.cc:482] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:17:03.936946 9618 raft_consensus.cc:3037] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:03.939648 9618 raft_consensus.cc:504] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45645 } } peers { permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 } } peers { permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 } }
I20260811 22:17:03.940286 9618 leader_election.cc:288] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [CANDIDATE]: Term 1 election: Requested vote from peers 8249a50251794775978676eb852393b9 (127.10.173.129:41449), 5532ce3adc024dd98dd648e9b2f2a749 (127.10.173.131:39251)
I20260811 22:17:03.940827 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dd93ce76cf463680b469e2d55eb37d" candidate_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8249a50251794775978676eb852393b9"
I20260811 22:17:03.940840 9520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dd93ce76cf463680b469e2d55eb37d" candidate_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5532ce3adc024dd98dd648e9b2f2a749"
I20260811 22:17:03.941062 9125 raft_consensus.cc:3037] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:03.941090 9520 raft_consensus.cc:3037] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:03.943565 9520 raft_consensus.cc:2454] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b6c7ef1e6034a8580f4d14de59ae24d in term 1.
I20260811 22:17:03.943750 9125 raft_consensus.cc:2454] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b6c7ef1e6034a8580f4d14de59ae24d in term 1.
I20260811 22:17:03.944072 9230 leader_election.cc:302] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5532ce3adc024dd98dd648e9b2f2a749, 6b6c7ef1e6034a8580f4d14de59ae24d; no voters:
I20260811 22:17:03.944411 9618 raft_consensus.cc:2781] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:03.944945 9618 raft_consensus.cc:686] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 1 LEADER]: Becoming Leader. State: Replica: 6b6c7ef1e6034a8580f4d14de59ae24d, State: Running, Role: LEADER
I20260811 22:17:03.945479 9618 consensus_queue.cc:237] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45645 } } peers { permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 } } peers { permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 } }
I20260811 22:17:03.950541 8922 catalog_manager.cc:5495] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b6c7ef1e6034a8580f4d14de59ae24d (127.10.173.130). New cstate: current_term: 1 leader_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b6c7ef1e6034a8580f4d14de59ae24d" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 45645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 } health_report { overall_health: UNKNOWN } } }
W20260811 22:17:03.993151 9191 tablet.cc:2369] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:17:04.022918 9604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:04.017268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34884 (local address 127.10.173.130:45645)
0811 22:17:04.017478 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:04.017483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:04.017502 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:17:04.017723 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:04.017727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:04.017830 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:04.017969 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:04.018373 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:04.019196 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:04.020634 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:04.021056 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:04.021165 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:04.021741 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:04.022279 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:04.022392 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:04.022623 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 22:17:04.022698 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:17:04.031965 9125 raft_consensus.cc:1261] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9 [term 1 FOLLOWER]: Refusing update from remote peer 6b6c7ef1e6034a8580f4d14de59ae24d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:17:04.031965 9520 raft_consensus.cc:1261] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749 [term 1 FOLLOWER]: Refusing update from remote peer 6b6c7ef1e6034a8580f4d14de59ae24d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:17:04.033206 9626 consensus_queue.cc:1035] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8249a50251794775978676eb852393b9" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 41449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:17:04.033560 9618 consensus_queue.cc:1035] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5532ce3adc024dd98dd648e9b2f2a749" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 39251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:17:04.046865 9636 mvcc.cc:205] Tried to move back new op lower bound from 7317449212023382016 to 7317449211685203968. Current Snapshot: MvccSnapshot[applied={T|T < 7317449212023382016}]
I20260811 22:17:04.058666 8921 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:17:04.058970 8921 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:17:04.063046 8921 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d10c18b41474b969afb7549a30eba4d: Sending DeleteTablet for 3 replicas of tablet 89dd93ce76cf463680b469e2d55eb37d
I20260811 22:17:04.064325 9474 tablet_service.cc:1514] Processing DeleteTablet for tablet 89dd93ce76cf463680b469e2d55eb37d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:17:04 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:49858
I20260811 22:17:04.064427 9099 tablet_service.cc:1514] Processing DeleteTablet for tablet 89dd93ce76cf463680b469e2d55eb37d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:17:04 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:35324
I20260811 22:17:04.064932 9290 tablet_service.cc:1514] Processing DeleteTablet for tablet 89dd93ce76cf463680b469e2d55eb37d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:17:04 UTC) from {username='kudu', principal='kudu/127.10.173.190@KRBTEST.COM'} at 127.0.0.1:34878
I20260811 22:17:04.071341 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 9014
I20260811 22:17:04.073220 9648 tablet_replica.cc:330] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: stopping tablet replica
I20260811 22:17:04.074122 9647 tablet_replica.cc:330] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: stopping tablet replica
I20260811 22:17:04.074491 9647 raft_consensus.cc:2229] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:17:04.075024 9647 raft_consensus.cc:2258] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:17:04.075417 9648 raft_consensus.cc:2229] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:17:04.076004 9648 raft_consensus.cc:2258] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:17:04.077056 9646 tablet_replica.cc:330] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: stopping tablet replica
I20260811 22:17:04.077471 9646 raft_consensus.cc:2229] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:17:04.077973 9646 raft_consensus.cc:2258] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:17:04.078267 9647 ts_tablet_manager.cc:1905] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:17:04.078554 9648 ts_tablet_manager.cc:1905] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:17:04.080070 9646 ts_tablet_manager.cc:1905] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:17:04.085121 9647 ts_tablet_manager.cc:1918] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:17:04.085274 9647 log.cc:1196] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-0/wal/wals/89dd93ce76cf463680b469e2d55eb37d
I20260811 22:17:04.085847 9647 ts_tablet_manager.cc:1939] T 89dd93ce76cf463680b469e2d55eb37d P 8249a50251794775978676eb852393b9: Deleting consensus metadata
I20260811 22:17:04.086494 9648 ts_tablet_manager.cc:1918] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:17:04.086779 9648 log.cc:1196] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/89dd93ce76cf463680b469e2d55eb37d
I20260811 22:17:04.086934 9646 ts_tablet_manager.cc:1918] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:17:04.087110 9646 log.cc:1196] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/89dd93ce76cf463680b469e2d55eb37d
I20260811 22:17:04.087563 9648 ts_tablet_manager.cc:1939] T 89dd93ce76cf463680b469e2d55eb37d P 6b6c7ef1e6034a8580f4d14de59ae24d: Deleting consensus metadata
I20260811 22:17:04.087599 9646 ts_tablet_manager.cc:1939] T 89dd93ce76cf463680b469e2d55eb37d P 5532ce3adc024dd98dd648e9b2f2a749: Deleting consensus metadata
I20260811 22:17:04.089432 8902 catalog_manager.cc:4841] TS 6b6c7ef1e6034a8580f4d14de59ae24d (127.10.173.130:45645): tablet 89dd93ce76cf463680b469e2d55eb37d (table test-table [id=d6e7bfefc7264518b5c36aad6000a8f9]) successfully deleted
I20260811 22:17:04.089454 8904 catalog_manager.cc:4841] TS 5532ce3adc024dd98dd648e9b2f2a749 (127.10.173.131:39251): tablet 89dd93ce76cf463680b469e2d55eb37d (table test-table [id=d6e7bfefc7264518b5c36aad6000a8f9]) successfully deleted
I20260811 22:17:04.090684 8902 catalog_manager.cc:4841] TS 8249a50251794775978676eb852393b9 (127.10.173.129:41449): tablet 89dd93ce76cf463680b469e2d55eb37d (table test-table [id=d6e7bfefc7264518b5c36aad6000a8f9]) successfully deleted
Aug 11 22:17:04 dist-test-slave-68xk krb5kdc[8858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.129@KRBTEST.COM
I20260811 22:17:04.106909 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:04.073436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.10.173.129:41449)
0811 22:17:04.073643 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:04.073649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:04.076899 (+ 3250us) server_negotiation.cc:407] Connection header received
0811 22:17:04.089667 (+ 12768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:04.089674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:04.089764 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:04.089881 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:04.092268 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:04.093260 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:04.094987 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:04.095629 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:04.095796 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:04.100330 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:04.100343 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:04.100356 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:04.100392 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:04.103989 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:04.104728 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:04.104735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:04.104738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:04.104879 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:04.105297 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:04.105301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:04.105303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:04.105661 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:04.105988 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:04.106292 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 22:17:04.106551 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:17:04.107633 9184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:04.198925 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 9014
I20260811 22:17:04.226941 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 9198
Aug 11 22:17:04 dist-test-slave-68xk krb5kdc[8858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.130@KRBTEST.COM
I20260811 22:17:04.246635 9604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:04.228087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34898 (local address 127.10.173.130:45645)
0811 22:17:04.228284 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:04.228289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:04.228313 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:17:04.228493 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:04.228498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:04.228639 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:04.228736 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:04.230519 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:04.231654 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:04.233565 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:04.234085 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:04.234207 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:04.239638 (+ 5431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:04.239654 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:04.239676 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:04.239728 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:04.243298 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:04.244105 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:04.244114 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:04.244116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:04.244276 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:04.244717 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:04.244724 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:04.244727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:04.245167 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:04.245576 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:04.245873 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 22:17:04.246149 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:17:04.247390 9377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:04.368105 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 9198
I20260811 22:17:04.396802 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 9394
Aug 11 22:17:04 dist-test-slave-68xk krb5kdc[8858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.131@KRBTEST.COM
I20260811 22:17:04.412853 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:04.397751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.10.173.131:39251)
0811 22:17:04.397931 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:04.397936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:04.397989 (+ 53us) server_negotiation.cc:407] Connection header received
0811 22:17:04.398217 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:04.398222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:04.398287 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:04.398386 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:04.399785 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:04.400566 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:04.401961 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:04.402396 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:04.402597 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:04.407213 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:04.407230 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:04.407247 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:04.407287 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:04.410021 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:04.410785 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:04.410793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:04.410795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:04.410943 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:04.411380 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:04.411383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:04.411385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:04.411759 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:04.411985 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:04.412394 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 22:17:04.412574 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:17:04.413607 9574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:04.504503 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 9394
I20260811 22:17:04.532392 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 8877
I20260811 22:17:04.533612 8967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:04.650801 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 8877
2026-08-11T22:17:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5113 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T22:17:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:04Z Disabled control of system clock
I20260811 22:17:04.757192 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:44859
--webserver_interface=127.10.173.190
--webserver_port=0
--builtin_ntp_servers=127.10.173.148:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:44859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:17:05.111764 9661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:05.112076 9661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:05.112162 9661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:05.119428 9661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:05.119558 9661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:05.119597 9661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:05.119649 9661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:05.119688 9661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:05.128978 9661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:44859
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:44859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:05.130336 9661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:05.132177 9661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:05.144901 9670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:05.144842 9672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:05.144861 9674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:05.145979 9661 server_base.cc:1034] running on GCE node
I20260811 22:17:05.146888 9661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:05.148339 9661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:05.149554 9661 hybrid_clock.cc:648] HybridClock initialized: now 1786486625149495 us; error 57 us; skew 500 ppm
I20260811 22:17:05.149955 9661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:17:05.152464 9661 webserver.cc:460] Webserver started at http://127.10.173.190:40803/ using document root <none> and password file <none>
I20260811 22:17:05.153175 9661 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:05.153293 9661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:05.153653 9661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:05.156777 9661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "8a40599739534566a7c3cc3d6a23b10f"
format_stamp: "Formatted at 2026-08-11 22:17:05 on dist-test-slave-68xk"
server_key: "60634ad7857c742c6afa6d6b72fb0b5d"
server_key_iv: "000bfdc40e38e191c14d489c14c27ecd"
server_key_version: "encryptionkey@0"
I20260811 22:17:05.157627 9661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "8a40599739534566a7c3cc3d6a23b10f"
format_stamp: "Formatted at 2026-08-11 22:17:05 on dist-test-slave-68xk"
server_key: "60634ad7857c742c6afa6d6b72fb0b5d"
server_key_iv: "000bfdc40e38e191c14d489c14c27ecd"
server_key_version: "encryptionkey@0"
I20260811 22:17:05.162839 9661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 22:17:05.166391 9682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:05.167917 9661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:17:05.168123 9661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "8a40599739534566a7c3cc3d6a23b10f"
format_stamp: "Formatted at 2026-08-11 22:17:05 on dist-test-slave-68xk"
server_key: "60634ad7857c742c6afa6d6b72fb0b5d"
server_key_iv: "000bfdc40e38e191c14d489c14c27ecd"
server_key_version: "encryptionkey@0"
I20260811 22:17:05.168354 9661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:05.184600 9661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:05.185436 9661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:05.185745 9661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:05.206593 9661 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:44859
I20260811 22:17:05.206682 9754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:44859 every 8 connection(s)
I20260811 22:17:05.208137 9661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:05.212291 9755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:05.217511 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 9661
I20260811 22:17:05.217730 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:05.218225 10934 external_mini_cluster.cc:1428] Setting key 4a4960fdaf565e0640d0474158d12177
I20260811 22:17:05.224321 9755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f: Bootstrap starting.
I20260811 22:17:05.227687 9755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:05.229039 9755 log.cc:824] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:05.232156 9762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:05.220256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37934 (local address 127.10.173.190:44859)
0811 22:17:05.221388 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:05.221402 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:05.221447 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:17:05.223266 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:05.223290 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:05.223738 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:05.224112 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:05.224674 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:05.225968 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:05.227403 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:05.228175 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:05.228385 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:05.229627 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:05.229641 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:05.229664 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:05.230480 (+ 816us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:05.230579 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:05.230607 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:05.230735 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:05.230960 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 22:17:05.231268 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":404,"threads_started":1}
I20260811 22:17:05.232414 9755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f: No bootstrap required, opened a new log
I20260811 22:17:05.241206 9755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40599739534566a7c3cc3d6a23b10f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44859 } }
I20260811 22:17:05.241564 9755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:05.241647 9755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a40599739534566a7c3cc3d6a23b10f, State: Initialized, Role: FOLLOWER
I20260811 22:17:05.242223 9755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40599739534566a7c3cc3d6a23b10f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44859 } }
I20260811 22:17:05.242503 9755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:05.242627 9755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:05.242764 9755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:05.245476 9755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40599739534566a7c3cc3d6a23b10f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44859 } }
I20260811 22:17:05.245959 9755 leader_election.cc:302] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a40599739534566a7c3cc3d6a23b10f; no voters:
I20260811 22:17:05.246634 9755 leader_election.cc:288] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:05.246793 9765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:05.247673 9765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [term 1 LEADER]: Becoming Leader. State: Replica: 8a40599739534566a7c3cc3d6a23b10f, State: Running, Role: LEADER
I20260811 22:17:05.248175 9765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40599739534566a7c3cc3d6a23b10f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44859 } }
I20260811 22:17:05.248947 9755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:05.251852 9767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a40599739534566a7c3cc3d6a23b10f. Latest consensus state: current_term: 1 leader_uuid: "8a40599739534566a7c3cc3d6a23b10f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40599739534566a7c3cc3d6a23b10f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44859 } } }
I20260811 22:17:05.252345 9767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:05.252959 9766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a40599739534566a7c3cc3d6a23b10f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a40599739534566a7c3cc3d6a23b10f" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44859 } } }
I20260811 22:17:05.253296 9766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:05.256605 9777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:05.261721 9777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:05.270637 9777 catalog_manager.cc:1347] Generated new cluster ID: c302b4226794481d820195c0c582ad22
I20260811 22:17:05.270797 9777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:05.306522 9777 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:05.308038 9777 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:05.314484 9777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f: Generated new TSK 0
I20260811 22:17:05.315588 9777 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:05.329044 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.129:0
--local_ip_for_outbound_sockets=127.10.173.129
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44859
--builtin_ntp_servers=127.10.173.148:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:17:05.684110 9796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:05.684381 9796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:05.684470 9796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:05.691630 9796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:05.691728 9796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:05.691833 9796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.129
I20260811 22:17:05.700868 9796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.173.129
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.129
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:05.702349 9796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:05.704610 9796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:05.718149 9802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:05.718209 9806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:05.718269 9803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:05.718922 9796 server_base.cc:1034] running on GCE node
I20260811 22:17:05.719547 9796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:05.720427 9796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:05.721669 9796 hybrid_clock.cc:648] HybridClock initialized: now 1786486625721627 us; error 71 us; skew 500 ppm
I20260811 22:17:05.721988 9796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:17:05.724438 9796 webserver.cc:460] Webserver started at http://127.10.173.129:45725/ using document root <none> and password file <none>
I20260811 22:17:05.725168 9796 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:05.725308 9796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:05.725659 9796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:05.728541 9796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/data/instance:
uuid: "339bfadff3bc40fbb2fce8d295a86dcf"
format_stamp: "Formatted at 2026-08-11 22:17:05 on dist-test-slave-68xk"
server_key: "8c69052e07da04142e53cab9f5ad94b8"
server_key_iv: "5bf15940e98f317404addc98a66accb9"
server_key_version: "encryptionkey@0"
I20260811 22:17:05.729396 9796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance:
uuid: "339bfadff3bc40fbb2fce8d295a86dcf"
format_stamp: "Formatted at 2026-08-11 22:17:05 on dist-test-slave-68xk"
server_key: "8c69052e07da04142e53cab9f5ad94b8"
server_key_iv: "5bf15940e98f317404addc98a66accb9"
server_key_version: "encryptionkey@0"
I20260811 22:17:05.734822 9796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:17:05.738354 9814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:05.739781 9796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:17:05.739976 9796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/wal
uuid: "339bfadff3bc40fbb2fce8d295a86dcf"
format_stamp: "Formatted at 2026-08-11 22:17:05 on dist-test-slave-68xk"
server_key: "8c69052e07da04142e53cab9f5ad94b8"
server_key_iv: "5bf15940e98f317404addc98a66accb9"
server_key_version: "encryptionkey@0"
I20260811 22:17:05.740249 9796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:05.757218 9796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:05.758035 9796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:05.758359 9796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:05.759366 9796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:17:05.760799 9796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:17:05.760939 9796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:05.761086 9796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:17:05.761180 9796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:05.796182 9796 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.129:43517
I20260811 22:17:05.796304 9971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.129:43517 every 8 connection(s)
I20260811 22:17:05.797549 9796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/data/info.pb
I20260811 22:17:05.802819 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 9796
I20260811 22:17:05.802994 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-0/wal/instance
I20260811 22:17:05.803457 10934 external_mini_cluster.cc:1428] Setting key a6432f042df02e3e0479e093df87be92
I20260811 22:17:05.806349 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.130:0
--local_ip_for_outbound_sockets=127.10.173.130
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44859
--builtin_ntp_servers=127.10.173.148:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:17:05.811072 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:05.800474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:48331 (local address 127.10.173.190:44859)
0811 22:17:05.800956 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:05.800964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:05.802164 (+ 1200us) server_negotiation.cc:407] Connection header received
0811 22:17:05.804196 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:05.804202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:05.804299 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:05.804400 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:05.805680 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:05.806505 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:05.808219 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:05.808628 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:05.808768 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:05.810177 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:05.810182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:05.810185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:05.810203 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:05.810245 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:05.810258 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:05.810357 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:05.810634 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 22:17:05.810737 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":230,"threads_started":1}
I20260811 22:17:05.812141 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:05.801088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:44859 (local address 127.10.173.129:48331)
0811 22:17:05.802000 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:05.802043 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:05.803913 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:05.804676 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:05.804684 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:05.804771 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:05.805497 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:05.805522 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:05.806714 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:05.806718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:05.808064 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:05.808079 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:05.808862 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:05.808871 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:05.809113 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:05.809848 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:17:05.809851 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:05.809890 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:17:05.809969 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:05.810358 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:05.810373 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:05.810383 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:17:05.810617 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 22:17:05.811037 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":277,"threads_started":1}
I20260811 22:17:05.813951 9972 heartbeater.cc:347] Connected to a master server at 127.10.173.190:44859
I20260811 22:17:05.814309 9972 heartbeater.cc:464] Registering TS with master...
I20260811 22:17:05.815220 9972 heartbeater.cc:511] Master 127.10.173.190:44859 requested a full tablet report, sending...
I20260811 22:17:05.817737 9706 ts_manager.cc:194] Registered new tserver with Master: 339bfadff3bc40fbb2fce8d295a86dcf (127.10.173.129:43517)
I20260811 22:17:05.819778 9706 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.129:48331
W20260811 22:17:06.160198 9976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:06.160523 9976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:06.160696 9976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:06.167807 9976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:06.167945 9976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:06.168134 9976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.130
I20260811 22:17:06.177079 9976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.173.130
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.130
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:06.178442 9976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:06.180276 9976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:06.193881 9985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:06.193806 9986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:06.193765 9988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:06.195551 9976 server_base.cc:1034] running on GCE node
I20260811 22:17:06.196141 9976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:06.197026 9976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:06.198228 9976 hybrid_clock.cc:648] HybridClock initialized: now 1786486626198188 us; error 38 us; skew 500 ppm
I20260811 22:17:06.198604 9976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:17:06.200951 9976 webserver.cc:460] Webserver started at http://127.10.173.130:34955/ using document root <none> and password file <none>
I20260811 22:17:06.201694 9976 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:06.201809 9976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:06.202150 9976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:06.205098 9976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/data/instance:
uuid: "4234b20db0e04b37929a1864a53cc5fe"
format_stamp: "Formatted at 2026-08-11 22:17:06 on dist-test-slave-68xk"
server_key: "c00836e95f873946782695bc52de3931"
server_key_iv: "04bb2fddd78b743a79d040d28bcb3470"
server_key_version: "encryptionkey@0"
I20260811 22:17:06.205950 9976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance:
uuid: "4234b20db0e04b37929a1864a53cc5fe"
format_stamp: "Formatted at 2026-08-11 22:17:06 on dist-test-slave-68xk"
server_key: "c00836e95f873946782695bc52de3931"
server_key_iv: "04bb2fddd78b743a79d040d28bcb3470"
server_key_version: "encryptionkey@0"
I20260811 22:17:06.211198 9976 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 22:17:06.214830 9995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:06.216215 9976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:17:06.216393 9976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/wal
uuid: "4234b20db0e04b37929a1864a53cc5fe"
format_stamp: "Formatted at 2026-08-11 22:17:06 on dist-test-slave-68xk"
server_key: "c00836e95f873946782695bc52de3931"
server_key_iv: "04bb2fddd78b743a79d040d28bcb3470"
server_key_version: "encryptionkey@0"
I20260811 22:17:06.216609 9976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:06.239106 9976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:06.239976 9976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:06.240309 9976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:06.241367 9976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:17:06.243050 9976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:17:06.243189 9976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:06.243317 9976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:17:06.243405 9976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:06.278739 9976 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.130:39911
I20260811 22:17:06.278856 10159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.130:39911 every 8 connection(s)
I20260811 22:17:06.280200 9976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/data/info.pb
I20260811 22:17:06.283324 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 9976
I20260811 22:17:06.283520 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/wal/instance
I20260811 22:17:06.283996 10934 external_mini_cluster.cc:1428] Setting key ea221cc375ad136c520cbf9678f4131b
I20260811 22:17:06.286943 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.173.131:0
--local_ip_for_outbound_sockets=127.10.173.131
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44859
--builtin_ntp_servers=127.10.173.148:37027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:17:06.295235 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.283315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.130:48275 (local address 127.10.173.190:44859)
0811 22:17:06.283574 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:06.283582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:06.285367 (+ 1785us) server_negotiation.cc:407] Connection header received
0811 22:17:06.287489 (+ 2122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:06.287497 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:06.287590 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:06.287705 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:06.288880 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.290025 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.291933 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.292626 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.292770 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.294219 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:06.294228 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:06.294235 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:06.294261 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:06.294303 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:06.294321 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:06.294443 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:06.294693 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 22:17:06.294828 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:17:06.296382 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.283909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:44859 (local address 127.10.173.130:48275)
0811 22:17:06.285182 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:06.285233 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:06.287172 (+ 1939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:06.287919 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:06.287928 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:06.288020 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:06.288701 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.288719 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.290210 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.290214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.291780 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.291794 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.292843 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.292848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.293074 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.293878 (+ 804us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:17:06.293881 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:06.293921 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:17:06.294008 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:06.294474 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:06.294485 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:06.294491 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:17:06.294653 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 22:17:06.295233 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":421,"threads_started":1}
I20260811 22:17:06.297988 10161 heartbeater.cc:347] Connected to a master server at 127.10.173.190:44859
I20260811 22:17:06.298324 10161 heartbeater.cc:464] Registering TS with master...
I20260811 22:17:06.299271 10161 heartbeater.cc:511] Master 127.10.173.190:44859 requested a full tablet report, sending...
I20260811 22:17:06.301106 9706 ts_manager.cc:194] Registered new tserver with Master: 4234b20db0e04b37929a1864a53cc5fe (127.10.173.130:39911)
I20260811 22:17:06.301946 9706 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.130:48275
W20260811 22:17:06.634137 10166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:06.634403 10166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:06.634605 10166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:06.641722 10166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:06.641821 10166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:06.642014 10166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.173.131
I20260811 22:17:06.652626 10166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.173.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.173.131
--webserver_port=0
--tserver_master_addrs=127.10.173.190:44859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.173.131
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:06.653949 10166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:06.655790 10166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:06.668838 10175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:06.668887 10172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:06.668915 10173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:06.669687 10166 server_base.cc:1034] running on GCE node
I20260811 22:17:06.670310 10166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:06.671257 10166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:06.672490 10166 hybrid_clock.cc:648] HybridClock initialized: now 1786486626672440 us; error 59 us; skew 500 ppm
I20260811 22:17:06.672856 10166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 22:17:06.675297 10166 webserver.cc:460] Webserver started at http://127.10.173.131:36905/ using document root <none> and password file <none>
I20260811 22:17:06.676000 10166 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:06.676141 10166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:06.676496 10166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:06.679512 10166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/data/instance:
uuid: "6a3d08a89f9b49daa7ea8a594649d537"
format_stamp: "Formatted at 2026-08-11 22:17:06 on dist-test-slave-68xk"
server_key: "c6aa601ba1c06d979aac4b0b5df1288f"
server_key_iv: "f567a2d7e226169e9fb55b823fcea7cf"
server_key_version: "encryptionkey@0"
I20260811 22:17:06.680344 10166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance:
uuid: "6a3d08a89f9b49daa7ea8a594649d537"
format_stamp: "Formatted at 2026-08-11 22:17:06 on dist-test-slave-68xk"
server_key: "c6aa601ba1c06d979aac4b0b5df1288f"
server_key_iv: "f567a2d7e226169e9fb55b823fcea7cf"
server_key_version: "encryptionkey@0"
I20260811 22:17:06.685832 10166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:17:06.689669 10181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:06.691457 10166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:17:06.691666 10166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/wal
uuid: "6a3d08a89f9b49daa7ea8a594649d537"
format_stamp: "Formatted at 2026-08-11 22:17:06 on dist-test-slave-68xk"
server_key: "c6aa601ba1c06d979aac4b0b5df1288f"
server_key_iv: "f567a2d7e226169e9fb55b823fcea7cf"
server_key_version: "encryptionkey@0"
I20260811 22:17:06.691910 10166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:06.742353 10166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:06.743312 10166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:06.743645 10166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:06.744763 10166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:17:06.746388 10166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:17:06.746541 10166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:06.746685 10166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:17:06.746776 10166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:06.782733 10166 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.131:40547
I20260811 22:17:06.782825 10350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.131:40547 every 8 connection(s)
I20260811 22:17:06.784130 10166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/data/info.pb
I20260811 22:17:06.791616 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 10166
I20260811 22:17:06.791838 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/wal/instance
I20260811 22:17:06.792383 10934 external_mini_cluster.cc:1428] Setting key ec804a318bea47bdb086612177db02a5
I20260811 22:17:06.799208 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.787173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.131:33697 (local address 127.10.173.190:44859)
0811 22:17:06.787388 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:06.787393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:06.788847 (+ 1454us) server_negotiation.cc:407] Connection header received
0811 22:17:06.791115 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:06.791120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:06.791195 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:06.791293 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:06.792832 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.793830 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.795881 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.796533 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.796680 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.798329 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:06.798337 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:06.798343 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:06.798364 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:06.798403 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:06.798441 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:06.798538 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:06.798783 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 22:17:06.798888 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:17:06.800215 10354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.787729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:44859 (local address 127.10.173.131:33697)
0811 22:17:06.788621 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:06.788686 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:06.790812 (+ 2126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:06.791615 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:06.791627 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:06.791759 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:06.792610 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.792638 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.793997 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.794002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.795726 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.795744 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.796851 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.796857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.797199 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.797998 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:17:06.798001 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:06.798039 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:17:06.798118 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:06.798565 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:06.798574 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:06.798580 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:17:06.798728 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 22:17:06.799041 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":268,"threads_started":1}
I20260811 22:17:06.801601 10351 heartbeater.cc:347] Connected to a master server at 127.10.173.190:44859
I20260811 22:17:06.801955 10351 heartbeater.cc:464] Registering TS with master...
I20260811 22:17:06.802791 10351 heartbeater.cc:511] Master 127.10.173.190:44859 requested a full tablet report, sending...
I20260811 22:17:06.804209 9706 ts_manager.cc:194] Registered new tserver with Master: 6a3d08a89f9b49daa7ea8a594649d537 (127.10.173.131:40547)
I20260811 22:17:06.804996 9706 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.173.131:33697
I20260811 22:17:06.808145 10934 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:17:06.818540 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.812612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.10.173.190:44859)
0811 22:17:06.812846 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:06.812850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:06.813319 (+ 469us) server_negotiation.cc:407] Connection header received
0811 22:17:06.813450 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:06.813454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:06.813529 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:06.813618 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:06.814107 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.814927 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.816292 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.816742 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.816858 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.817893 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:06.817898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:06.817901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:06.817918 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:06.817952 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:06.817965 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:06.818044 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:06.818137 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 22:17:06.818214 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:17:06.823632 9972 heartbeater.cc:503] Master 127.10.173.190:44859 was elected leader, sending a full tablet report...
I20260811 22:17:06.822788 9706 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:17:06.830902 9706 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:17:06.852952 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.843753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.10.173.129:43517)
0811 22:17:06.844596 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:06.844606 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:06.844812 (+ 206us) server_negotiation.cc:407] Connection header received
0811 22:17:06.845148 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:06.845157 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:06.845490 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:06.845701 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:17:06.846730 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.847967 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.850370 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.851675 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.851902 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.852146 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:06.852382 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 22:17:06.852555 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":487,"threads_started":1}
I20260811 22:17:06.852939 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.843561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43517 (local address 127.0.0.1:41716)
0811 22:17:06.844655 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:06.844706 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:06.844904 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:06.845990 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:06.845997 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:06.846034 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:17:06.846516 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.846532 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.848112 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.848119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.850119 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.850137 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.851900 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.851906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.852112 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.852168 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 22:17:06.852336 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 22:17:06.852528 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":671,"threads_started":1}
I20260811 22:17:06.864034 9882 tablet_service.cc:1467] Processing CreateTablet for tablet 9e984f17135b4e3d825074b6db3a4665 (DEFAULT_TABLE table=test-table [id=9c8f5b145b2a45baaeb3a5d9416b2e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:17:06.865757 9882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e984f17135b4e3d825074b6db3a4665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:06.867054 10373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.845156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:39911 (local address 127.0.0.1:51874)
0811 22:17:06.845851 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:06.845870 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:06.845990 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:06.859669 (+ 13679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:06.859675 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:06.859702 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:17:06.860207 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.860317 (+ 110us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.862184 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.862192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.864199 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.864214 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.866111 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.866119 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.866333 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.866376 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:17:06.866478 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:17:06.866632 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"mutex_wait_us":27,"thread_start_us":290,"threads_started":1}
I20260811 22:17:06.867659 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.845492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.10.173.130:39911)
0811 22:17:06.858960 (+ 13468us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:06.858970 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:06.859004 (+ 34us) server_negotiation.cc:407] Connection header received
0811 22:17:06.859111 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:06.859119 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:06.859426 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:06.860623 (+ 1197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:17:06.860906 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.862016 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.864421 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.865837 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.866741 (+ 904us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.866976 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:06.867071 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 22:17:06.867261 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13282,"thread_start_us":13135,"threads_started":1}
I20260811 22:17:06.868848 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.845877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:40547 (local address 127.0.0.1:59138)
0811 22:17:06.853185 (+ 7308us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:06.853201 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:06.853304 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:06.859921 (+ 6617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:06.859926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:06.859954 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:17:06.860603 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.860618 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.863081 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.863089 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.865151 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.865170 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.867918 (+ 2748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.867925 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.868137 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.868173 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:17:06.868328 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 22:17:06.868486 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7205,"mutex_wait_us":28,"thread_start_us":13174,"threads_started":1}
I20260811 22:17:06.871773 10378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.846613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59138 (local address 127.10.173.131:40547)
0811 22:17:06.858962 (+ 12349us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:06.858970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:06.859003 (+ 33us) server_negotiation.cc:407] Connection header received
0811 22:17:06.859458 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:06.859469 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:06.859692 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:06.860668 (+ 976us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:17:06.861004 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.862839 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.865591 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.867653 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.867999 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.868217 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:06.868382 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 22:17:06.868549 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12190,"thread_start_us":12030,"threads_started":1}
I20260811 22:17:06.871224 10069 tablet_service.cc:1467] Processing CreateTablet for tablet 9e984f17135b4e3d825074b6db3a4665 (DEFAULT_TABLE table=test-table [id=9c8f5b145b2a45baaeb3a5d9416b2e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:17:06.873539 10069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e984f17135b4e3d825074b6db3a4665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:06.875962 10255 tablet_service.cc:1467] Processing CreateTablet for tablet 9e984f17135b4e3d825074b6db3a4665 (DEFAULT_TABLE table=test-table [id=9c8f5b145b2a45baaeb3a5d9416b2e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:17:06.877545 10255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e984f17135b4e3d825074b6db3a4665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:06.879379 10379 tablet_bootstrap.cc:492] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf: Bootstrap starting.
I20260811 22:17:06.883805 10379 tablet_bootstrap.cc:654] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:06.884933 10379 log.cc:824] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:06.887830 10379 tablet_bootstrap.cc:492] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf: No bootstrap required, opened a new log
I20260811 22:17:06.888163 10379 ts_tablet_manager.cc:1397] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 22:17:06.890091 10384 tablet_bootstrap.cc:492] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: Bootstrap starting.
I20260811 22:17:06.893214 10386 tablet_bootstrap.cc:492] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: Bootstrap starting.
I20260811 22:17:06.895267 10384 tablet_bootstrap.cc:654] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:06.896720 10384 log.cc:824] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:06.896780 10386 tablet_bootstrap.cc:654] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:06.898216 10386 log.cc:824] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:06.900463 10384 tablet_bootstrap.cc:492] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: No bootstrap required, opened a new log
I20260811 22:17:06.900779 10384 ts_tablet_manager.cc:1397] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 22:17:06.901100 10386 tablet_bootstrap.cc:492] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: No bootstrap required, opened a new log
I20260811 22:17:06.901366 10386 ts_tablet_manager.cc:1397] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 22:17:06.901116 10379 raft_consensus.cc:348] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43517 } } peers { permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 } } peers { permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 } }
I20260811 22:17:06.901609 10379 raft_consensus.cc:374] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:06.901714 10379 raft_consensus.cc:729] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339bfadff3bc40fbb2fce8d295a86dcf, State: Initialized, Role: FOLLOWER
I20260811 22:17:06.902376 10379 consensus_queue.cc:260] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43517 } } peers { permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 } } peers { permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 } }
I20260811 22:17:06.912842 10386 raft_consensus.cc:348] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43517 } } peers { permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 } } peers { permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 } }
I20260811 22:17:06.913493 10386 raft_consensus.cc:374] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:06.913636 10386 raft_consensus.cc:729] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3d08a89f9b49daa7ea8a594649d537, State: Initialized, Role: FOLLOWER
I20260811 22:17:06.914439 10386 consensus_queue.cc:260] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43517 } } peers { permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 } } peers { permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 } }
I20260811 22:17:06.914762 10384 raft_consensus.cc:348] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43517 } } peers { permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 } } peers { permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 } }
I20260811 22:17:06.915251 10384 raft_consensus.cc:374] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:06.915362 10384 raft_consensus.cc:729] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4234b20db0e04b37929a1864a53cc5fe, State: Initialized, Role: FOLLOWER
I20260811 22:17:06.915904 10351 heartbeater.cc:503] Master 127.10.173.190:44859 was elected leader, sending a full tablet report...
I20260811 22:17:06.916384 10386 ts_tablet_manager.cc:1428] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260811 22:17:06.916630 10390 raft_consensus.cc:482] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:17:06.916908 10390 raft_consensus.cc:504] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43517 } } peers { permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 } } peers { permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 } }
I20260811 22:17:06.918957 10390 leader_election.cc:288] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4234b20db0e04b37929a1864a53cc5fe (127.10.173.130:39911), 6a3d08a89f9b49daa7ea8a594649d537 (127.10.173.131:40547)
I20260811 22:17:06.920020 10384 consensus_queue.cc:260] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43517 } } peers { permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 } } peers { permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 } }
I20260811 22:17:06.920851 10379 ts_tablet_manager.cc:1428] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf: Time spent starting tablet: real 0.033s user 0.015s sys 0.015s
I20260811 22:17:06.927310 10161 heartbeater.cc:503] Master 127.10.173.190:44859 was elected leader, sending a full tablet report...
I20260811 22:17:06.928231 10384 ts_tablet_manager.cc:1428] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: Time spent starting tablet: real 0.027s user 0.013s sys 0.011s
I20260811 22:17:06.931402 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.919624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:40385 (local address 127.10.173.130:39911)
0811 22:17:06.919823 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:06.919829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:06.923443 (+ 3614us) server_negotiation.cc:407] Connection header received
0811 22:17:06.923781 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:06.923789 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:06.923915 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:06.923994 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:17:06.924995 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.926504 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.929232 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.930386 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.930587 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.930754 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:06.931005 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 22:17:06.931136 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":77}
I20260811 22:17:06.931728 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.919997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.130:39911 (local address 127.10.173.129:40385)
0811 22:17:06.923315 (+ 3318us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:06.923341 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:06.923576 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:06.924240 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:06.924247 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:06.924283 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:17:06.924766 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.924782 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.926750 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.926758 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.929024 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.929037 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.930615 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.930622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.930820 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.930857 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:17:06.930970 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 22:17:06.931118 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3134,"thread_start_us":2962,"threads_started":1}
I20260811 22:17:06.932749 10097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e984f17135b4e3d825074b6db3a4665" candidate_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4234b20db0e04b37929a1864a53cc5fe" is_pre_election: true
I20260811 22:17:06.933209 10097 raft_consensus.cc:2454] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 339bfadff3bc40fbb2fce8d295a86dcf in term 0.
I20260811 22:17:06.933837 9817 leader_election.cc:302] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339bfadff3bc40fbb2fce8d295a86dcf, 4234b20db0e04b37929a1864a53cc5fe; no voters:
I20260811 22:17:06.934223 10389 raft_consensus.cc:2781] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:17:06.934345 10389 raft_consensus.cc:482] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:17:06.934522 10389 raft_consensus.cc:3037] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:06.935717 10378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.923800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.173.129:54131 (local address 127.10.173.131:40547)
0811 22:17:06.924052 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:06.924061 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:06.929023 (+ 4962us) server_negotiation.cc:407] Connection header received
0811 22:17:06.929270 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:06.929276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:06.929356 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:06.929469 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:17:06.930057 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.931243 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.933603 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.934759 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.934941 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.935017 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:06.935333 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 22:17:06.935467 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:17:06.935967 10395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.923603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.131:40547 (local address 127.10.173.129:54131)
0811 22:17:06.928899 (+ 5296us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:06.928920 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:06.929053 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:06.929500 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:06.929505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:06.929531 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:17:06.929890 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.929900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.931413 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.931421 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.933395 (+ 1974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:06.933410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.934977 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.934984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:06.935158 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.935187 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:17:06.935304 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:17:06.935452 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5163,"thread_start_us":4993,"threads_started":1}
I20260811 22:17:06.937026 10281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e984f17135b4e3d825074b6db3a4665" candidate_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a3d08a89f9b49daa7ea8a594649d537" is_pre_election: true
I20260811 22:17:06.937350 10389 raft_consensus.cc:504] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43517 } } peers { permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 } } peers { permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 } }
I20260811 22:17:06.937537 10281 raft_consensus.cc:2454] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 339bfadff3bc40fbb2fce8d295a86dcf in term 0.
I20260811 22:17:06.938084 10389 leader_election.cc:288] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [CANDIDATE]: Term 1 election: Requested vote from peers 4234b20db0e04b37929a1864a53cc5fe (127.10.173.130:39911), 6a3d08a89f9b49daa7ea8a594649d537 (127.10.173.131:40547)
I20260811 22:17:06.938549 10097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e984f17135b4e3d825074b6db3a4665" candidate_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4234b20db0e04b37929a1864a53cc5fe"
I20260811 22:17:06.938820 10097 raft_consensus.cc:3037] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:06.939913 10281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e984f17135b4e3d825074b6db3a4665" candidate_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a3d08a89f9b49daa7ea8a594649d537"
I20260811 22:17:06.940127 10281 raft_consensus.cc:3037] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:06.941462 10097 raft_consensus.cc:2454] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 339bfadff3bc40fbb2fce8d295a86dcf in term 1.
I20260811 22:17:06.941915 9817 leader_election.cc:302] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339bfadff3bc40fbb2fce8d295a86dcf, 4234b20db0e04b37929a1864a53cc5fe; no voters:
I20260811 22:17:06.942206 10389 raft_consensus.cc:2781] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:06.942404 10389 raft_consensus.cc:686] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 1 LEADER]: Becoming Leader. State: Replica: 339bfadff3bc40fbb2fce8d295a86dcf, State: Running, Role: LEADER
I20260811 22:17:06.943082 10281 raft_consensus.cc:2454] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 339bfadff3bc40fbb2fce8d295a86dcf in term 1.
I20260811 22:17:06.943306 10389 consensus_queue.cc:237] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43517 } } peers { permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 } } peers { permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 } }
I20260811 22:17:06.948125 9706 catalog_manager.cc:5495] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 339bfadff3bc40fbb2fce8d295a86dcf (127.10.173.129). New cstate: current_term: 1 leader_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "339bfadff3bc40fbb2fce8d295a86dcf" member_type: VOTER last_known_addr { host: "127.10.173.129" port: 43517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 } health_report { overall_health: UNKNOWN } } }
I20260811 22:17:06.980194 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:06.974453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.10.173.129:43517)
0811 22:17:06.974677 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:06.974682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:06.974700 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:17:06.974890 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:06.974894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:06.975002 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:06.975128 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:06.975541 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.976273 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.977874 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:06.978302 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:06.978402 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:06.978982 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:06.979561 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:06.979673 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:06.979804 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 22:17:06.979869 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:17:06.989064 10097 raft_consensus.cc:1261] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe [term 1 FOLLOWER]: Refusing update from remote peer 339bfadff3bc40fbb2fce8d295a86dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:17:06.989174 10281 raft_consensus.cc:1261] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537 [term 1 FOLLOWER]: Refusing update from remote peer 339bfadff3bc40fbb2fce8d295a86dcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:17:06.990054 10390 consensus_queue.cc:1035] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "4234b20db0e04b37929a1864a53cc5fe" member_type: VOTER last_known_addr { host: "127.10.173.130" port: 39911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:17:06.990453 10389 consensus_queue.cc:1035] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a3d08a89f9b49daa7ea8a594649d537" member_type: VOTER last_known_addr { host: "127.10.173.131" port: 40547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:17:07.002781 10400 mvcc.cc:205] Tried to move back new op lower bound from 7317449224137134080 to 7317449223964745728. Current Snapshot: MvccSnapshot[applied={T|T < 7317449224137134080}]
I20260811 22:17:07.020375 9705 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:17:07.020715 9705 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:17:07.024765 9705 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a40599739534566a7c3cc3d6a23b10f: Sending DeleteTablet for 3 replicas of tablet 9e984f17135b4e3d825074b6db3a4665
I20260811 22:17:07.025861 9882 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e984f17135b4e3d825074b6db3a4665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:17:07 UTC) from {username='slave'} at 127.0.0.1:41716
I20260811 22:17:07.026165 10255 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e984f17135b4e3d825074b6db3a4665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:17:07 UTC) from {username='slave'} at 127.0.0.1:59138
I20260811 22:17:07.026194 10069 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e984f17135b4e3d825074b6db3a4665 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:17:07 UTC) from {username='slave'} at 127.0.0.1:51874
I20260811 22:17:07.028031 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 9796
I20260811 22:17:07.059020 10420 tablet_replica.cc:330] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf: stopping tablet replica
I20260811 22:17:07.059690 10420 raft_consensus.cc:2229] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:17:07.060393 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:07.030338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41732 (local address 127.10.173.129:43517)
0811 22:17:07.030533 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:07.030540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:07.039199 (+ 8659us) server_negotiation.cc:407] Connection header received
0811 22:17:07.039410 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:07.039422 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:07.039552 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:07.039664 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:07.048020 (+ 8356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:07.049365 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:07.055008 (+ 5643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:07.055669 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:07.055784 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:07.058398 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:07.058446 (+ 48us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:07.058464 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:07.059324 (+ 860us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:07.059398 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:07.059430 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:07.059523 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:07.059857 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 22:17:07.059973 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:17:07.060740 10420 raft_consensus.cc:2258] T 9e984f17135b4e3d825074b6db3a4665 P 339bfadff3bc40fbb2fce8d295a86dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:17:07.061843 9967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:07.064399 10422 tablet_replica.cc:330] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: stopping tablet replica
I20260811 22:17:07.064877 10422 raft_consensus.cc:2229] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:17:07.065157 10421 tablet_replica.cc:330] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: stopping tablet replica
I20260811 22:17:07.065486 10422 raft_consensus.cc:2258] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:17:07.065845 10421 raft_consensus.cc:2229] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:17:07.066377 10421 raft_consensus.cc:2258] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:17:07.067935 10422 ts_tablet_manager.cc:1905] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:17:07.069209 10421 ts_tablet_manager.cc:1905] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:17:07.073482 10422 ts_tablet_manager.cc:1918] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:17:07.073673 10422 log.cc:1196] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-2/wal/wals/9e984f17135b4e3d825074b6db3a4665
I20260811 22:17:07.074147 10422 ts_tablet_manager.cc:1939] T 9e984f17135b4e3d825074b6db3a4665 P 6a3d08a89f9b49daa7ea8a594649d537: Deleting consensus metadata
I20260811 22:17:07.075910 9688 catalog_manager.cc:4841] TS 6a3d08a89f9b49daa7ea8a594649d537 (127.10.173.131:40547): tablet 9e984f17135b4e3d825074b6db3a4665 (table test-table [id=9c8f5b145b2a45baaeb3a5d9416b2e73]) successfully deleted
I20260811 22:17:07.077332 10421 ts_tablet_manager.cc:1918] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:17:07.077528 10421 log.cc:1196] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: Deleting WAL directory at /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486285256683-10934-0/minicluster-data/ts-1/wal/wals/9e984f17135b4e3d825074b6db3a4665
I20260811 22:17:07.078246 10421 ts_tablet_manager.cc:1939] T 9e984f17135b4e3d825074b6db3a4665 P 4234b20db0e04b37929a1864a53cc5fe: Deleting consensus metadata
I20260811 22:17:07.080374 9688 catalog_manager.cc:4841] TS 4234b20db0e04b37929a1864a53cc5fe (127.10.173.130:39911): tablet 9e984f17135b4e3d825074b6db3a4665 (table test-table [id=9c8f5b145b2a45baaeb3a5d9416b2e73]) successfully deleted
I20260811 22:17:07.168027 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 9796
W20260811 22:17:07.197397 9688 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.173.129:43517 (error 108)
I20260811 22:17:07.198180 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 9976
I20260811 22:17:07.199601 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:07.198990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43517 (local address 127.0.0.1:41736)
0811 22:17:07.199349 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:07.199451 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211}
W20260811 22:17:07.199893 9688 catalog_manager.cc:4584] TS 339bfadff3bc40fbb2fce8d295a86dcf (127.10.173.129:43517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9e984f17135b4e3d825074b6db3a4665: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43517: connect: Connection refused (error 111)
I20260811 22:17:07.205838 10374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:07.199125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51878 (local address 127.10.173.130:39911)
0811 22:17:07.199641 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:07.199646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:07.199661 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:17:07.199716 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:07.199748 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:07.199853 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:07.199984 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:07.200427 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:07.201187 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:07.202611 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:07.203127 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:07.203275 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:07.204383 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:07.204392 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:07.204406 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:07.205016 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:07.205088 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:07.205108 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:07.205240 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:07.205379 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 22:17:07.205503 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389}
I20260811 22:17:07.206609 10152 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:07.253643 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:07.253044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43517 (local address 127.0.0.1:41746)
0811 22:17:07.253327 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:07.253425 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 22:17:07.254807 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:07.254395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43517 (local address 127.0.0.1:41760)
0811 22:17:07.254600 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:07.254670 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 22:17:07.305436 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 9976
I20260811 22:17:07.315634 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:07.314998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43517 (local address 127.0.0.1:41772)
0811 22:17:07.315297 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:07.315399 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260811 22:17:07.317221 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:07.316618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43517 (local address 127.0.0.1:41784)
0811 22:17:07.316915 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:07.317021 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260811 22:17:07.335829 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 10166
I20260811 22:17:07.342820 10378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:07.336783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.10.173.131:40547)
0811 22:17:07.336966 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:07.336971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:07.336990 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:17:07.337138 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:07.337143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:07.337231 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:07.337318 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:07.337704 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:07.338531 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:07.339827 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:07.340217 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:07.340312 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:07.341404 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:07.341413 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:07.341428 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:07.342019 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:07.342067 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:07.342088 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:07.342181 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:07.342347 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 22:17:07.342497 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:17:07.343516 10344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:07.398208 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:07.397670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43517 (local address 127.0.0.1:41790)
0811 22:17:07.397931 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:07.398049 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 22:17:07.399443 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:07.398979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.129:43517 (local address 127.0.0.1:41802)
0811 22:17:07.399225 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:07.399327 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.129:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260811 22:17:07.433815 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 10166
I20260811 22:17:07.460508 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 9661
I20260811 22:17:07.461714 9750 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:07.573088 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 9661
2026-08-11T22:17:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2916 ms)
[----------] 6 tests from EncryptionPolicyTest (24731 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:07 dist-test-slave-68xk krb5kdc[10439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:07 dist-test-slave-68xk krb5kdc[10439](info): set up 2 sockets
Aug 11 22:17:07 dist-test-slave-68xk krb5kdc[10439](info): commencing operation
krb5kdc: starting...
W20260811 22:17:09.001276 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.377s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:09 dist-test-slave-68xk krb5kdc[10439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:09.330518 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:09.680264 10459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:09.680583 10459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:09.680756 10459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:09.688330 10459 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:09.688513 10459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:09.688625 10459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:09.688746 10459 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:09.688827 10459 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:09.699008 10459 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35107
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:09.700462 10459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:09.701906 10459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:09.715447 10467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:09.715449 10472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:09.715474 10468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:09.716451 10459 server_base.cc:1034] running on GCE node
I20260811 22:17:09.717310 10459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:09.718914 10459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:09.720155 10459 hybrid_clock.cc:648] HybridClock initialized: now 1786486629720084 us; error 62 us; skew 500 ppm
I20260811 22:17:09.720517 10459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:09 dist-test-slave-68xk krb5kdc[10439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486629, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:09.727208 10459 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:09.728734 10459 webserver.cc:460] Webserver started at http://127.0.0.1:46723/ using document root <none> and password file <none>
I20260811 22:17:09.729430 10459 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:09.729581 10459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:09.729910 10459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:09.733029 10459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "a799fd0feaa14db19c3cb1a5acf4f320"
format_stamp: "Formatted at 2026-08-11 22:17:09 on dist-test-slave-68xk"
server_key: "c91747d5df00c99949b58c0793f3cb4d"
server_key_iv: "eaba165de20fdc09828d35c8a76d5ca7"
server_key_version: "encryptionkey@0"
I20260811 22:17:09.733822 10459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "a799fd0feaa14db19c3cb1a5acf4f320"
format_stamp: "Formatted at 2026-08-11 22:17:09 on dist-test-slave-68xk"
server_key: "c91747d5df00c99949b58c0793f3cb4d"
server_key_iv: "eaba165de20fdc09828d35c8a76d5ca7"
server_key_version: "encryptionkey@0"
I20260811 22:17:09.739010 10459 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 22:17:09.742592 10482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:09.743853 10459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:17:09.744040 10459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "a799fd0feaa14db19c3cb1a5acf4f320"
format_stamp: "Formatted at 2026-08-11 22:17:09 on dist-test-slave-68xk"
server_key: "c91747d5df00c99949b58c0793f3cb4d"
server_key_iv: "eaba165de20fdc09828d35c8a76d5ca7"
server_key_version: "encryptionkey@0"
I20260811 22:17:09.744233 10459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:09.766355 10459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:09.770390 10459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:09.770750 10459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:09.790763 10459 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35107
I20260811 22:17:09.790850 10555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35107 every 8 connection(s)
I20260811 22:17:09.792163 10459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:09.793337 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 10459
I20260811 22:17:09.793592 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:09.794459 10934 external_mini_cluster.cc:1428] Setting key e33d6dfff52ae3b3639fa62db9d9e167
I20260811 22:17:09.797225 10556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:09.808913 10556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320: Bootstrap starting.
Aug 11 22:17:09 dist-test-slave-68xk krb5kdc[10439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:17:09.812674 10556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:09.813794 10556 log.cc:824] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:09.817313 10556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320: No bootstrap required, opened a new log
I20260811 22:17:09.818688 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:09.796628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38315 (local address 127.0.0.1:35107)
0811 22:17:09.797462 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:09.797474 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:09.797515 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:17:09.799244 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:09.799266 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:09.799654 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:09.800056 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:09.801619 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:09.803036 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:09.804673 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:09.805414 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:09.805650 (+ 236us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:09.810595 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:09.810604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:09.810622 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:09.810661 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:09.813877 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:09.814998 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:09.815007 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:09.815023 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:09.815257 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:09.815871 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:09.815875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:09.815877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:09.816430 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:09.816664 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:09.817259 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 22:17:09.817611 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":559,"thread_start_us":323,"threads_started":1}
I20260811 22:17:09.822715 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:09.825378 10556 raft_consensus.cc:348] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a799fd0feaa14db19c3cb1a5acf4f320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } }
I20260811 22:17:09.825796 10556 raft_consensus.cc:374] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:09.825918 10556 raft_consensus.cc:729] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a799fd0feaa14db19c3cb1a5acf4f320, State: Initialized, Role: FOLLOWER
I20260811 22:17:09.826638 10556 consensus_queue.cc:260] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a799fd0feaa14db19c3cb1a5acf4f320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } }
I20260811 22:17:09.826875 10556 raft_consensus.cc:388] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:09.827005 10556 raft_consensus.cc:482] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:09.827198 10556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:09.829870 10556 raft_consensus.cc:504] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a799fd0feaa14db19c3cb1a5acf4f320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } }
I20260811 22:17:09.830477 10556 leader_election.cc:302] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a799fd0feaa14db19c3cb1a5acf4f320; no voters:
I20260811 22:17:09.831211 10556 leader_election.cc:288] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:09.831329 10570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:09.832495 10570 raft_consensus.cc:686] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [term 1 LEADER]: Becoming Leader. State: Replica: a799fd0feaa14db19c3cb1a5acf4f320, State: Running, Role: LEADER
I20260811 22:17:09.833144 10570 consensus_queue.cc:237] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a799fd0feaa14db19c3cb1a5acf4f320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } }
I20260811 22:17:09.833710 10556 sys_catalog.cc:564] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:09.836264 10572 sys_catalog.cc:455] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a799fd0feaa14db19c3cb1a5acf4f320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a799fd0feaa14db19c3cb1a5acf4f320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } } }
I20260811 22:17:09.836751 10572 sys_catalog.cc:458] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:09.839016 10573 sys_catalog.cc:455] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a799fd0feaa14db19c3cb1a5acf4f320. Latest consensus state: current_term: 1 leader_uuid: "a799fd0feaa14db19c3cb1a5acf4f320" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a799fd0feaa14db19c3cb1a5acf4f320" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } } }
I20260811 22:17:09.839490 10573 sys_catalog.cc:458] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:09.839318 10576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:09.845628 10576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:09.846360 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:09.827893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57887 (local address 127.0.0.1:35107)
0811 22:17:09.828091 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:09.828099 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:09.828453 (+ 354us) server_negotiation.cc:407] Connection header received
0811 22:17:09.828674 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:09.828679 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:09.828760 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:09.828880 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:09.830370 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:09.831333 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:09.833217 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:09.833726 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:09.833885 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:09.837200 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:09.837210 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:09.837217 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:09.837268 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:09.841527 (+ 4259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:09.842736 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:09.842742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:09.842745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:09.843001 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:09.843613 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:09.843617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:09.843620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:09.844026 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:09.844341 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:09.845792 (+ 1451us) server_negotiation.cc:299] Negotiation successful
0811 22:17:09.845986 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:17:09.855940 10576 catalog_manager.cc:1347] Generated new cluster ID: 9812b661d967442f934a947ea6477e40
I20260811 22:17:09.856065 10576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:09.878664 10576 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:09.880555 10576 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:09.897051 10576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a799fd0feaa14db19c3cb1a5acf4f320: Generated new TSK 0
I20260811 22:17:09.897955 10576 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:09.908581 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 10459
I20260811 22:17:09.909930 10550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:10.003485 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 10459
W20260811 22:17:10.031966 10437 connection.cc:535] client connection to 127.0.0.1:35107 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35107): Connection reset by peer (error 104)
2026-08-11T22:17:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:10 dist-test-slave-68xk krb5kdc[10599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:10 dist-test-slave-68xk krb5kdc[10599](info): set up 2 sockets
Aug 11 22:17:10 dist-test-slave-68xk krb5kdc[10599](info): commencing operation
krb5kdc: starting...
W20260811 22:17:11.403514 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.352s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:11 dist-test-slave-68xk krb5kdc[10599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:11.746465 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41055
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:12.103822 10617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:12.104141 10617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:12.104238 10617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:12.111433 10617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:12.111548 10617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:12.111603 10617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:12.111671 10617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:12.111717 10617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:12.121658 10617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:12.122959 10617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:12.124251 10617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:12.136456 10631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:12.136421 10630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:12.136518 10634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:12.137468 10617 server_base.cc:1034] running on GCE node
I20260811 22:17:12.138253 10617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:12.139691 10617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:12.140905 10617 hybrid_clock.cc:648] HybridClock initialized: now 1786486632140865 us; error 33 us; skew 500 ppm
I20260811 22:17:12.141286 10617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:12 dist-test-slave-68xk krb5kdc[10599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486632, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:12.144682 10617 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:12.146142 10617 webserver.cc:460] Webserver started at http://127.0.0.1:39441/ using document root <none> and password file <none>
I20260811 22:17:12.146898 10617 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:12.147012 10617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:12.147289 10617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:12.150553 10617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "0e7798ddecfd43a0a80313e63e504612"
format_stamp: "Formatted at 2026-08-11 22:17:12 on dist-test-slave-68xk"
server_key: "54f236f679c33a855d0a696d48dbf666"
server_key_iv: "a2565e7b17e8c85a2589098d912f1412"
server_key_version: "encryptionkey@0"
I20260811 22:17:12.151365 10617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "0e7798ddecfd43a0a80313e63e504612"
format_stamp: "Formatted at 2026-08-11 22:17:12 on dist-test-slave-68xk"
server_key: "54f236f679c33a855d0a696d48dbf666"
server_key_iv: "a2565e7b17e8c85a2589098d912f1412"
server_key_version: "encryptionkey@0"
I20260811 22:17:12.157110 10617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 22:17:12.160832 10650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:12.162328 10617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:17:12.162585 10617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "0e7798ddecfd43a0a80313e63e504612"
format_stamp: "Formatted at 2026-08-11 22:17:12 on dist-test-slave-68xk"
server_key: "54f236f679c33a855d0a696d48dbf666"
server_key_iv: "a2565e7b17e8c85a2589098d912f1412"
server_key_version: "encryptionkey@0"
I20260811 22:17:12.162779 10617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:12.178363 10617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:12.182863 10617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:12.183121 10617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:12.203649 10617 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41055
I20260811 22:17:12.203688 10724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41055 every 8 connection(s)
I20260811 22:17:12.205176 10617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:12.209291 10725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:12.214218 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 10617
I20260811 22:17:12.214397 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:12.214926 10934 external_mini_cluster.cc:1428] Setting key 7ed81cdc53e910af7720434762f1dc4c
I20260811 22:17:12.221834 10725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612: Bootstrap starting.
I20260811 22:17:12.225899 10725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:12.227388 10725 log.cc:824] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:12.230861 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:12.217051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38527 (local address 127.0.0.1:41055)
0811 22:17:12.217979 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:12.217992 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:12.218045 (+ 53us) server_negotiation.cc:407] Connection header received
0811 22:17:12.220226 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:12.220251 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:12.220690 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:12.221023 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:12.221789 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:12.223324 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:12.225462 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:12.226341 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:12.226606 (+ 265us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:12.228062 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:12.228075 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:12.228096 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:12.228977 (+ 881us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:12.229086 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:12.229115 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:12.229233 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:12.229618 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 22:17:12.229804 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":435,"threads_started":1}
I20260811 22:17:12.231014 10725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612: No bootstrap required, opened a new log
I20260811 22:17:12.235055 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:12.242211 10725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7798ddecfd43a0a80313e63e504612" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41055 } }
I20260811 22:17:12.242822 10725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:12.242983 10725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e7798ddecfd43a0a80313e63e504612, State: Initialized, Role: FOLLOWER
I20260811 22:17:12.243737 10725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7798ddecfd43a0a80313e63e504612" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41055 } }
I20260811 22:17:12.244058 10725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:12.244218 10725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:12.244416 10725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:12.247947 10725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7798ddecfd43a0a80313e63e504612" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41055 } }
I20260811 22:17:12.248322 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:12.240889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33929 (local address 127.0.0.1:41055)
0811 22:17:12.241097 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:12.241104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:12.241450 (+ 346us) server_negotiation.cc:407] Connection header received
0811 22:17:12.241633 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:12.241636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:12.241693 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:12.241785 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:12.242540 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:12.243457 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:12.245554 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:12.246032 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:12.246129 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:12.247421 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:12.247425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:12.247429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:12.247444 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:12.247503 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:12.247518 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:12.247598 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:12.247864 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 22:17:12.247908 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:17:12.248674 10725 leader_election.cc:302] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e7798ddecfd43a0a80313e63e504612; no voters:
I20260811 22:17:12.249439 10725 leader_election.cc:288] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:12.249591 10737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:12.250473 10737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [term 1 LEADER]: Becoming Leader. State: Replica: 0e7798ddecfd43a0a80313e63e504612, State: Running, Role: LEADER
I20260811 22:17:12.251173 10737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7798ddecfd43a0a80313e63e504612" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41055 } }
I20260811 22:17:12.252096 10725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:12.254631 10739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e7798ddecfd43a0a80313e63e504612. Latest consensus state: current_term: 1 leader_uuid: "0e7798ddecfd43a0a80313e63e504612" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7798ddecfd43a0a80313e63e504612" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41055 } } }
I20260811 22:17:12.254812 10738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e7798ddecfd43a0a80313e63e504612" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e7798ddecfd43a0a80313e63e504612" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41055 } } }
I20260811 22:17:12.255182 10739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:12.255192 10738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:12.258912 10744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:12.263793 10744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:12.274312 10744 catalog_manager.cc:1347] Generated new cluster ID: 724935f129c648bdaddc5c06f2d4b6b5
I20260811 22:17:12.274467 10744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:12.286881 10744 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:12.288419 10744 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:12.306844 10744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e7798ddecfd43a0a80313e63e504612: Generated new TSK 0
I20260811 22:17:12.307744 10744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:12.314859 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 10617
W20260811 22:17:12.315970 10719 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38527
W20260811 22:17:12.316784 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 10617: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:12.317018 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 10617
W20260811 22:17:12.345914 10597 connection.cc:535] client connection to 127.0.0.1:41055 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41055): Connection reset by peer (error 104)
2026-08-11T22:17:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:12 dist-test-slave-68xk krb5kdc[10763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:12 dist-test-slave-68xk krb5kdc[10763](info): set up 2 sockets
Aug 11 22:17:12 dist-test-slave-68xk krb5kdc[10763](info): commencing operation
krb5kdc: starting...
W20260811 22:17:13.683162 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:13 dist-test-slave-68xk krb5kdc[10763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:14.014611 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42599
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:14.356590 10789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:14.356880 10789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:14.356971 10789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:14.364159 10789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:14.364279 10789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:14.364326 10789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:14.364387 10789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:14.364434 10789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:14.374320 10789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42599
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:14.375584 10789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:14.376819 10789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:14.388622 10795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:14.388849 10798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:14.388899 10796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:14.389778 10789 server_base.cc:1034] running on GCE node
I20260811 22:17:14.390596 10789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:14.391950 10789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:14.393195 10789 hybrid_clock.cc:648] HybridClock initialized: now 1786486634393153 us; error 59 us; skew 500 ppm
I20260811 22:17:14.393546 10789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:14 dist-test-slave-68xk krb5kdc[10763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486634, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:14.396831 10789 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:14.398512 10789 webserver.cc:460] Webserver started at http://127.0.0.1:37011/ using document root <none> and password file <none>
I20260811 22:17:14.399212 10789 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:14.399367 10789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:14.399689 10789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:14.402805 10789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "dc0802c069634c2fb6f8f467f76d53c6"
format_stamp: "Formatted at 2026-08-11 22:17:14 on dist-test-slave-68xk"
server_key: "970041f3e9f1941d5dee986283fa1e9f"
server_key_iv: "dc1887d8d28bcde639b5ebc1d295f3ec"
server_key_version: "encryptionkey@0"
I20260811 22:17:14.403597 10789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "dc0802c069634c2fb6f8f467f76d53c6"
format_stamp: "Formatted at 2026-08-11 22:17:14 on dist-test-slave-68xk"
server_key: "970041f3e9f1941d5dee986283fa1e9f"
server_key_iv: "dc1887d8d28bcde639b5ebc1d295f3ec"
server_key_version: "encryptionkey@0"
I20260811 22:17:14.408810 10789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 22:17:14.412434 10808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:14.413769 10789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:17:14.413956 10789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "dc0802c069634c2fb6f8f467f76d53c6"
format_stamp: "Formatted at 2026-08-11 22:17:14 on dist-test-slave-68xk"
server_key: "970041f3e9f1941d5dee986283fa1e9f"
server_key_iv: "dc1887d8d28bcde639b5ebc1d295f3ec"
server_key_version: "encryptionkey@0"
I20260811 22:17:14.414155 10789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:14.420050 10789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:14.424542 10789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:14.424849 10789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:14.444384 10789 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42599
I20260811 22:17:14.444473 10883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42599 every 8 connection(s)
I20260811 22:17:14.445894 10789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:14.449713 10884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:14.455494 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 10789
I20260811 22:17:14.455704 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:14.456135 10934 external_mini_cluster.cc:1428] Setting key bd2a6bd9c3dbbe3777c4b248a9d034b5
I20260811 22:17:14.460736 10884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6: Bootstrap starting.
I20260811 22:17:14.464030 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:14.458036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60399 (local address 127.0.0.1:42599)
0811 22:17:14.458985 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:14.458996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:14.459037 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:17:14.460684 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:14.460706 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:14.461075 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:14.461374 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:14.462504 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:14.462510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:14.462525 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:14.462540 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:14.462603 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:14.462623 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:14.462806 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:14.462941 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 22:17:14.463153 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":433,"threads_started":1}
I20260811 22:17:14.464470 10884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:14.465643 10884 log.cc:824] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:14.467953 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:14.468905 10884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6: No bootstrap required, opened a new log
I20260811 22:17:14.476125 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:14.473251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43085 (local address 127.0.0.1:42599)
0811 22:17:14.473489 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:14.473498 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:14.474077 (+ 579us) server_negotiation.cc:407] Connection header received
0811 22:17:14.474296 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:14.474301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:14.474347 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:14.474477 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:14.475421 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:14.475427 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:14.475431 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:14.475459 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:14.475498 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:14.475519 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:14.475661 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:14.475771 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 22:17:14.475833 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:17:14.478293 10884 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0802c069634c2fb6f8f467f76d53c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } }
I20260811 22:17:14.478783 10884 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:14.478884 10884 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0802c069634c2fb6f8f467f76d53c6, State: Initialized, Role: FOLLOWER
I20260811 22:17:14.479590 10884 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0802c069634c2fb6f8f467f76d53c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } }
I20260811 22:17:14.479918 10884 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:14.480086 10884 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:14.480278 10884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:14.482623 10884 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0802c069634c2fb6f8f467f76d53c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } }
I20260811 22:17:14.483151 10884 leader_election.cc:302] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc0802c069634c2fb6f8f467f76d53c6; no voters:
I20260811 22:17:14.483769 10884 leader_election.cc:288] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:14.483927 10897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:14.484717 10897 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [term 1 LEADER]: Becoming Leader. State: Replica: dc0802c069634c2fb6f8f467f76d53c6, State: Running, Role: LEADER
I20260811 22:17:14.485291 10897 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0802c069634c2fb6f8f467f76d53c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } }
I20260811 22:17:14.486042 10884 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:14.488126 10901 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc0802c069634c2fb6f8f467f76d53c6. Latest consensus state: current_term: 1 leader_uuid: "dc0802c069634c2fb6f8f467f76d53c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0802c069634c2fb6f8f467f76d53c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } } }
I20260811 22:17:14.488703 10901 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:14.488627 10898 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc0802c069634c2fb6f8f467f76d53c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc0802c069634c2fb6f8f467f76d53c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42599 } } }
I20260811 22:17:14.488933 10898 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:14.491237 10905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:14.497056 10905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:14.506775 10905 catalog_manager.cc:1347] Generated new cluster ID: c4527b77fd12479a826e93050c35db9e
I20260811 22:17:14.506891 10905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:14.533414 10905 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:14.533938 10905 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:14.543984 10905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc0802c069634c2fb6f8f467f76d53c6: Generated new TSK 0
I20260811 22:17:14.544984 10905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:14.556419 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 10789
W20260811 22:17:14.557747 10878 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60399
W20260811 22:17:14.558624 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 10789: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:14.558832 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 10789
2026-08-11T22:17:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2244 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:14 dist-test-slave-68xk krb5kdc[10925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:14 dist-test-slave-68xk krb5kdc[10925](info): set up 2 sockets
Aug 11 22:17:14 dist-test-slave-68xk krb5kdc[10925](info): commencing operation
krb5kdc: starting...
W20260811 22:17:15.932534 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:16 dist-test-slave-68xk krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:16.255632 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36909
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:16.600060 10955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:16.600380 10955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:16.600557 10955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:16.608031 10955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:16.608186 10955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:16.608273 10955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:16.608412 10955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:16.608495 10955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:16.618672 10955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:16.620023 10955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:16.621352 10955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:16.634008 10964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:16.634053 10968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:16.634081 10965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:16.634979 10955 server_base.cc:1034] running on GCE node
I20260811 22:17:16.635774 10955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:16.637316 10955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:16.638522 10955 hybrid_clock.cc:648] HybridClock initialized: now 1786486636638473 us; error 48 us; skew 500 ppm
I20260811 22:17:16.638897 10955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:16 dist-test-slave-68xk krb5kdc[10925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486636, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:16.646890 10955 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:16.648444 10955 webserver.cc:460] Webserver started at http://127.0.0.1:37535/ using document root <none> and password file <none>
I20260811 22:17:16.649155 10955 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:16.649329 10955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:16.649658 10955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:16.652899 10955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "364cb453828544548ac34d99cbcc554e"
format_stamp: "Formatted at 2026-08-11 22:17:16 on dist-test-slave-68xk"
server_key: "1e22cefa5ee453cdd616943973e9e8b0"
server_key_iv: "f0c3bc1c333fce33f5e32fed82b21764"
server_key_version: "encryptionkey@0"
I20260811 22:17:16.653764 10955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "364cb453828544548ac34d99cbcc554e"
format_stamp: "Formatted at 2026-08-11 22:17:16 on dist-test-slave-68xk"
server_key: "1e22cefa5ee453cdd616943973e9e8b0"
server_key_iv: "f0c3bc1c333fce33f5e32fed82b21764"
server_key_version: "encryptionkey@0"
I20260811 22:17:16.659346 10955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:17:16.662990 10978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:16.664408 10955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 22:17:16.664608 10955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "364cb453828544548ac34d99cbcc554e"
format_stamp: "Formatted at 2026-08-11 22:17:16 on dist-test-slave-68xk"
server_key: "1e22cefa5ee453cdd616943973e9e8b0"
server_key_iv: "f0c3bc1c333fce33f5e32fed82b21764"
server_key_version: "encryptionkey@0"
I20260811 22:17:16.664822 10955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:16.691869 10955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:16.696239 10955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:16.696581 10955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:16.715667 10955 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36909
I20260811 22:17:16.715756 11046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36909 every 8 connection(s)
I20260811 22:17:16.717065 10955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:16.720968 11049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:16.726321 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 10955
I20260811 22:17:16.726539 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:16.727110 10934 external_mini_cluster.cc:1428] Setting key 3408e4d074ce79e7fc3cbe1359c3c29a
I20260811 22:17:16.732329 11049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e: Bootstrap starting.
I20260811 22:17:16.736111 11049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:16.737370 11049 log.cc:824] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:16.740273 11049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e: No bootstrap required, opened a new log
Aug 11 22:17:16 dist-test-slave-68xk krb5kdc[10925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:17:16.747961 11049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364cb453828544548ac34d99cbcc554e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } }
I20260811 22:17:16.748375 11049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:16.748517 11049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 364cb453828544548ac34d99cbcc554e, State: Initialized, Role: FOLLOWER
I20260811 22:17:16.749202 11049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364cb453828544548ac34d99cbcc554e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } }
I20260811 22:17:16.749473 11049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:16.749653 11049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:16.749859 11049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:16.750473 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:16.728914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50061 (local address 127.0.0.1:36909)
0811 22:17:16.729860 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:16.729871 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:16.729915 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:17:16.731669 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:16.731690 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:16.732076 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:16.732380 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:16.734299 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:16.735639 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:16.737524 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:16.738240 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:16.738468 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:16.742701 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:16.742708 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:16.742724 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:16.742767 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:16.745946 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:16.746739 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:16.746755 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:16.746761 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:16.746932 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:16.747342 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:16.747348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:16.747351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:16.747861 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:16.748181 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:16.748635 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 22:17:16.749186 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":471,"threads_started":1}
I20260811 22:17:16.752427 11049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364cb453828544548ac34d99cbcc554e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } }
I20260811 22:17:16.752966 11049 leader_election.cc:302] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 364cb453828544548ac34d99cbcc554e; no voters:
I20260811 22:17:16.753700 11049 leader_election.cc:288] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:16.753819 11057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:16.754372 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:16.755755 11049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:16.766170 11057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [term 1 LEADER]: Becoming Leader. State: Replica: 364cb453828544548ac34d99cbcc554e, State: Running, Role: LEADER
I20260811 22:17:16.766860 11057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364cb453828544548ac34d99cbcc554e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } }
I20260811 22:17:16.769896 11058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "364cb453828544548ac34d99cbcc554e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364cb453828544548ac34d99cbcc554e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } } }
I20260811 22:17:16.770166 11058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:16.774179 11060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 364cb453828544548ac34d99cbcc554e. Latest consensus state: current_term: 1 leader_uuid: "364cb453828544548ac34d99cbcc554e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "364cb453828544548ac34d99cbcc554e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36909 } } }
I20260811 22:17:16.774593 11060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:16.777998 11086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:16.783365 11086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:16.783926 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:16.765192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43543 (local address 127.0.0.1:36909)
0811 22:17:16.765383 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:16.765391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:16.766016 (+ 625us) server_negotiation.cc:407] Connection header received
0811 22:17:16.766112 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:16.766116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:16.766195 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:16.766288 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:16.768296 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:16.769358 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:16.771010 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:16.771659 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:16.771779 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:16.775736 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:16.775744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:16.775749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:16.775799 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:16.780291 (+ 4492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:16.781171 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:16.781176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:16.781179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:16.781338 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:16.781859 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:16.781871 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:16.781879 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:16.782359 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:16.782703 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:16.783126 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 22:17:16.783389 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:17:16.792337 11086 catalog_manager.cc:1347] Generated new cluster ID: 66e49c497e144667876c1ade579164b5
I20260811 22:17:16.792466 11086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:16.821592 11086 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:16.823540 11086 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:16.834846 11086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 364cb453828544548ac34d99cbcc554e: Generated new TSK 0
I20260811 22:17:16.835866 11086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:16.846676 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 10955
I20260811 22:17:16.847730 11040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:16.942123 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 10955
2026-08-11T22:17:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:17 dist-test-slave-68xk krb5kdc[11097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:17 dist-test-slave-68xk krb5kdc[11097](info): set up 2 sockets
Aug 11 22:17:17 dist-test-slave-68xk krb5kdc[11097](info): commencing operation
krb5kdc: starting...
W20260811 22:17:18.323683 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.335s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:18 dist-test-slave-68xk krb5kdc[11097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:18.650112 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36357
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:19.000476 11115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:19.000780 11115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:19.000962 11115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:19.011400 11115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:19.011572 11115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:19.011665 11115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:19.011756 11115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:19.011842 11115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:19.023031 11115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:19.024497 11115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:19.025939 11115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:19.038275 11122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:19.038343 11123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:19.038343 11125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:19.039598 11115 server_base.cc:1034] running on GCE node
I20260811 22:17:19.040442 11115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:19.041857 11115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:19.043099 11115 hybrid_clock.cc:648] HybridClock initialized: now 1786486639043041 us; error 50 us; skew 500 ppm
I20260811 22:17:19.043503 11115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:19 dist-test-slave-68xk krb5kdc[11097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486639, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:19.046826 11115 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:19.048403 11115 webserver.cc:460] Webserver started at http://127.0.0.1:37319/ using document root <none> and password file <none>
I20260811 22:17:19.049093 11115 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:19.049252 11115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:19.049573 11115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:19.052680 11115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "394c67a20e594b62b5205b3603d06712"
format_stamp: "Formatted at 2026-08-11 22:17:19 on dist-test-slave-68xk"
server_key: "383b2e857412fdec2fc353bf88dc97e7"
server_key_iv: "68e5667252e75888f45e49d50ee34a49"
server_key_version: "encryptionkey@0"
I20260811 22:17:19.053531 11115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "394c67a20e594b62b5205b3603d06712"
format_stamp: "Formatted at 2026-08-11 22:17:19 on dist-test-slave-68xk"
server_key: "383b2e857412fdec2fc353bf88dc97e7"
server_key_iv: "68e5667252e75888f45e49d50ee34a49"
server_key_version: "encryptionkey@0"
I20260811 22:17:19.058962 11115 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 22:17:19.062670 11135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:19.064116 11115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:17:19.064342 11115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "394c67a20e594b62b5205b3603d06712"
format_stamp: "Formatted at 2026-08-11 22:17:19 on dist-test-slave-68xk"
server_key: "383b2e857412fdec2fc353bf88dc97e7"
server_key_iv: "68e5667252e75888f45e49d50ee34a49"
server_key_version: "encryptionkey@0"
I20260811 22:17:19.064560 11115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:19.083922 11115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:19.088634 11115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:19.088991 11115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:19.109958 11115 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36357
I20260811 22:17:19.109993 11210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36357 every 8 connection(s)
I20260811 22:17:19.111433 11115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:19.112244 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11115
I20260811 22:17:19.112453 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:19.113078 10934 external_mini_cluster.cc:1428] Setting key 121104af5e38d7c605e97995a2f6bdcd
I20260811 22:17:19.116415 11212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:19.128665 11218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:19.115053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38695 (local address 127.0.0.1:36357)
0811 22:17:19.115842 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:19.115860 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:19.115910 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:17:19.118273 (+ 2363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:19.118299 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:19.118821 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:19.119282 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:19.119832 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:19.121900 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:19.123351 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:19.124350 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:19.124527 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:19.125720 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:19.125732 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:19.125751 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:19.126540 (+ 789us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:19.126634 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:19.126678 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:19.126798 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:19.127004 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 22:17:19.127308 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":332,"threads_started":1}
I20260811 22:17:19.130877 11212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712: Bootstrap starting.
I20260811 22:17:19.132674 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:19.134316 11212 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:19.135424 11212 log.cc:824] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:19.138816 11212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712: No bootstrap required, opened a new log
I20260811 22:17:19.143895 11218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:19.137158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47881 (local address 127.0.0.1:36357)
0811 22:17:19.137375 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:19.137381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:19.137790 (+ 409us) server_negotiation.cc:407] Connection header received
0811 22:17:19.137946 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:19.137953 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:19.138036 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:19.138145 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:19.138761 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:19.139851 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:19.141162 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:19.141884 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:19.142010 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:19.143072 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:19.143079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:19.143084 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:19.143105 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:19.143148 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:19.143169 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:19.143268 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:19.143459 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 22:17:19.143553 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:17:19.147585 11212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394c67a20e594b62b5205b3603d06712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36357 } }
I20260811 22:17:19.147927 11212 raft_consensus.cc:374] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:19.148001 11212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394c67a20e594b62b5205b3603d06712, State: Initialized, Role: FOLLOWER
I20260811 22:17:19.148468 11212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394c67a20e594b62b5205b3603d06712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36357 } }
I20260811 22:17:19.148666 11212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:19.148816 11212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:19.148994 11212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:19.151402 11212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394c67a20e594b62b5205b3603d06712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36357 } }
I20260811 22:17:19.151896 11212 leader_election.cc:302] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 394c67a20e594b62b5205b3603d06712; no voters:
I20260811 22:17:19.152469 11212 leader_election.cc:288] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:19.152621 11226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:19.153491 11226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [term 1 LEADER]: Becoming Leader. State: Replica: 394c67a20e594b62b5205b3603d06712, State: Running, Role: LEADER
I20260811 22:17:19.154177 11226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394c67a20e594b62b5205b3603d06712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36357 } }
I20260811 22:17:19.154585 11212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:19.157570 11229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 394c67a20e594b62b5205b3603d06712. Latest consensus state: current_term: 1 leader_uuid: "394c67a20e594b62b5205b3603d06712" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394c67a20e594b62b5205b3603d06712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36357 } } }
I20260811 22:17:19.157913 11229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:19.158452 11228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "394c67a20e594b62b5205b3603d06712" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394c67a20e594b62b5205b3603d06712" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36357 } } }
I20260811 22:17:19.158828 11228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:19.163415 11238 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:19.169382 11238 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:19.179073 11238 catalog_manager.cc:1347] Generated new cluster ID: 3da05f3896984f00b50d0a36ee9239ab
I20260811 22:17:19.179206 11238 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:19.189631 11238 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:19.190999 11238 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:19.200320 11238 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 394c67a20e594b62b5205b3603d06712: Generated new TSK 0
I20260811 22:17:19.201122 11238 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:19.215826 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11115
W20260811 22:17:19.216848 11204 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38695
W20260811 22:17:19.217603 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11115: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:19.217815 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11115
2026-08-11T22:17:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2274 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:19 dist-test-slave-68xk krb5kdc[11252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:19 dist-test-slave-68xk krb5kdc[11252](info): set up 2 sockets
Aug 11 22:17:19 dist-test-slave-68xk krb5kdc[11252](info): commencing operation
krb5kdc: starting...
W20260811 22:17:20.632218 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.369s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:20 dist-test-slave-68xk krb5kdc[11252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:20.962596 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:21.317548 11269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:21.317857 11269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:21.317952 11269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:21.325280 11269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:21.325397 11269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:21.325446 11269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:21.325510 11269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:21.325548 11269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:21.335647 11269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:21.336951 11269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:21.338379 11269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:21.350864 11276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:21.350807 11282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:21.350854 11279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:21.351765 11269 server_base.cc:1034] running on GCE node
I20260811 22:17:21.352622 11269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:21.354050 11269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:21.355288 11269 hybrid_clock.cc:648] HybridClock initialized: now 1786486641355242 us; error 60 us; skew 500 ppm
I20260811 22:17:21.355672 11269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:21 dist-test-slave-68xk krb5kdc[11252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486641, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:21.358892 11269 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:21.360464 11269 webserver.cc:460] Webserver started at http://127.0.0.1:36435/ using document root <none> and password file <none>
I20260811 22:17:21.361147 11269 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:21.361328 11269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:21.361644 11269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:21.364701 11269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c"
format_stamp: "Formatted at 2026-08-11 22:17:21 on dist-test-slave-68xk"
server_key: "1eb610d9d4e921371cb564df695246ff"
server_key_iv: "5a4c1b098187c89b6b07442dfc3767a7"
server_key_version: "encryptionkey@0"
I20260811 22:17:21.365511 11269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c"
format_stamp: "Formatted at 2026-08-11 22:17:21 on dist-test-slave-68xk"
server_key: "1eb610d9d4e921371cb564df695246ff"
server_key_iv: "5a4c1b098187c89b6b07442dfc3767a7"
server_key_version: "encryptionkey@0"
I20260811 22:17:21.370767 11269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:17:21.374526 11292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:21.376076 11269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:17:21.376277 11269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c"
format_stamp: "Formatted at 2026-08-11 22:17:21 on dist-test-slave-68xk"
server_key: "1eb610d9d4e921371cb564df695246ff"
server_key_iv: "5a4c1b098187c89b6b07442dfc3767a7"
server_key_version: "encryptionkey@0"
I20260811 22:17:21.376485 11269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:21.382318 11269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:21.386212 11269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:21.386611 11269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:21.406309 11269 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36675
I20260811 22:17:21.406353 11372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36675 every 8 connection(s)
I20260811 22:17:21.407799 11269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:21.411751 11376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:21.414230 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11269
I20260811 22:17:21.414453 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:21.415026 10934 external_mini_cluster.cc:1428] Setting key 349c3af3fec30b1d369f4ef543786cd5
I20260811 22:17:21.423108 11379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:21.416881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58221 (local address 127.0.0.1:36675)
0811 22:17:21.417763 (+ 882us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:21.417775 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:21.417823 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:17:21.419572 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:21.419596 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:21.419930 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:21.420257 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:21.421432 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:21.421439 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:21.421451 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:21.421465 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:21.421526 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:21.421548 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:21.421730 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:21.421844 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 22:17:21.422047 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":421,"threads_started":1}
I20260811 22:17:21.424451 11376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c: Bootstrap starting.
I20260811 22:17:21.426954 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:21.427973 11376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:21.429250 11376 log.cc:824] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:21.432233 11376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c: No bootstrap required, opened a new log
I20260811 22:17:21.435446 11379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:21.432489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52951 (local address 127.0.0.1:36675)
0811 22:17:21.432751 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:21.432760 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:21.433279 (+ 519us) server_negotiation.cc:407] Connection header received
0811 22:17:21.433486 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:21.433492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:21.433554 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:21.433659 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:21.434728 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:21.434733 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:21.434741 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:21.434767 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:21.434810 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:21.434826 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:21.434978 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:21.435080 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 22:17:21.435152 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:17:21.441256 11376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } }
I20260811 22:17:21.441681 11376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:21.441821 11376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae36ac20ad04a6daaed75f45c1a9f4c, State: Initialized, Role: FOLLOWER
I20260811 22:17:21.442530 11376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } }
I20260811 22:17:21.442864 11376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:21.443034 11376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:21.443251 11376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:21.445510 11376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } }
I20260811 22:17:21.445963 11376 leader_election.cc:302] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ae36ac20ad04a6daaed75f45c1a9f4c; no voters:
I20260811 22:17:21.446584 11376 leader_election.cc:288] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:21.446749 11389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:21.447424 11389 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [term 1 LEADER]: Becoming Leader. State: Replica: 0ae36ac20ad04a6daaed75f45c1a9f4c, State: Running, Role: LEADER
I20260811 22:17:21.448014 11389 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } }
I20260811 22:17:21.448760 11376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:21.451357 11390 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } } }
I20260811 22:17:21.451814 11390 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:21.452251 11391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ae36ac20ad04a6daaed75f45c1a9f4c. Latest consensus state: current_term: 1 leader_uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ae36ac20ad04a6daaed75f45c1a9f4c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36675 } } }
I20260811 22:17:21.452822 11391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:21.453955 11403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:21.459424 11403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:21.468497 11403 catalog_manager.cc:1347] Generated new cluster ID: 48ce29a877f24c9da6772c3b3d2281e1
I20260811 22:17:21.468623 11403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:21.498795 11403 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:21.499300 11403 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:21.513413 11403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ae36ac20ad04a6daaed75f45c1a9f4c: Generated new TSK 0
I20260811 22:17:21.514536 11403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:21.536155 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11269
W20260811 22:17:21.537437 11364 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58221
W20260811 22:17:21.538229 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11269: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:21.538400 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11269
2026-08-11T22:17:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:21 dist-test-slave-68xk krb5kdc[11426](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:21 dist-test-slave-68xk krb5kdc[11426](info): set up 2 sockets
Aug 11 22:17:21 dist-test-slave-68xk krb5kdc[11426](info): commencing operation
krb5kdc: starting...
W20260811 22:17:22.900496 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:23 dist-test-slave-68xk krb5kdc[11426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:23.237396 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41925
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41925
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:23.592628 11442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:23.593009 11442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:23.593207 11442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:23.600556 11442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:23.600739 11442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:23.600854 11442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:23.600967 11442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:23.601064 11442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:23.611094 11442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:23.612517 11442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:23.613899 11442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:23.626080 11452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:23.626034 11450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:23.626108 11456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:23.626891 11442 server_base.cc:1034] running on GCE node
I20260811 22:17:23.627871 11442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:23.629500 11442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:23.630771 11442 hybrid_clock.cc:648] HybridClock initialized: now 1786486643630737 us; error 58 us; skew 500 ppm
I20260811 22:17:23.631210 11442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:23 dist-test-slave-68xk krb5kdc[11426](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486643, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:23.634687 11442 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:23.636185 11442 webserver.cc:460] Webserver started at http://127.0.0.1:38881/ using document root <none> and password file <none>
I20260811 22:17:23.636833 11442 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:23.636936 11442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:23.637202 11442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:23.640311 11442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "120c5e38fe6b4ca29a7abc617839634f"
format_stamp: "Formatted at 2026-08-11 22:17:23 on dist-test-slave-68xk"
server_key: "dda6ad08bd5ea7d25da932b327b2fd3c"
server_key_iv: "3e21add07d5ba9e990285d5d33c061f5"
server_key_version: "encryptionkey@0"
I20260811 22:17:23.641062 11442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "120c5e38fe6b4ca29a7abc617839634f"
format_stamp: "Formatted at 2026-08-11 22:17:23 on dist-test-slave-68xk"
server_key: "dda6ad08bd5ea7d25da932b327b2fd3c"
server_key_iv: "3e21add07d5ba9e990285d5d33c061f5"
server_key_version: "encryptionkey@0"
I20260811 22:17:23.646374 11442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:17:23.649870 11468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:23.651500 11442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:17:23.651679 11442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "120c5e38fe6b4ca29a7abc617839634f"
format_stamp: "Formatted at 2026-08-11 22:17:23 on dist-test-slave-68xk"
server_key: "dda6ad08bd5ea7d25da932b327b2fd3c"
server_key_iv: "3e21add07d5ba9e990285d5d33c061f5"
server_key_version: "encryptionkey@0"
I20260811 22:17:23.651897 11442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:23.663546 11442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:23.667845 11442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:23.668092 11442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:23.687872 11442 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41925
I20260811 22:17:23.687943 11547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41925 every 8 connection(s)
I20260811 22:17:23.689746 11442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:23.690084 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11442
I20260811 22:17:23.690277 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:23.691015 10934 external_mini_cluster.cc:1428] Setting key f78c872297748df8778318990d98d716
I20260811 22:17:23.694339 11549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 22:17:23 dist-test-slave-68xk krb5kdc[11426](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:17:23.708222 11549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f: Bootstrap starting.
I20260811 22:17:23.711766 11549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:23.713042 11549 log.cc:824] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:23.716883 11549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f: No bootstrap required, opened a new log
I20260811 22:17:23.717317 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:23.693257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:43947 (local address 127.0.0.1:41925)
0811 22:17:23.694162 (+ 905us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:23.694175 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:23.694217 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:17:23.696089 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:23.696112 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:23.696497 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:23.696858 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:23.698722 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:23.700465 (+ 1743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:23.702353 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:23.703119 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:23.703254 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:23.708452 (+ 5198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:23.708462 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:23.708477 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:23.708512 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:23.711646 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:23.712580 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:23.712589 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:23.712598 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:23.712880 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:23.713608 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:23.713619 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:23.713624 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:23.714643 (+ 1019us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:23.714980 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:23.715721 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 22:17:23.716120 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":424,"threads_started":1}
I20260811 22:17:23.721422 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:23.725992 11549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120c5e38fe6b4ca29a7abc617839634f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41925 } }
I20260811 22:17:23.726560 11549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:23.726688 11549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 120c5e38fe6b4ca29a7abc617839634f, State: Initialized, Role: FOLLOWER
I20260811 22:17:23.727415 11549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120c5e38fe6b4ca29a7abc617839634f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41925 } }
I20260811 22:17:23.727690 11549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:23.727859 11549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:23.728046 11549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:23.731346 11549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120c5e38fe6b4ca29a7abc617839634f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41925 } }
I20260811 22:17:23.732003 11549 leader_election.cc:302] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 120c5e38fe6b4ca29a7abc617839634f; no voters:
I20260811 22:17:23.732638 11549 leader_election.cc:288] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:23.732812 11565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:23.734252 11565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [term 1 LEADER]: Becoming Leader. State: Replica: 120c5e38fe6b4ca29a7abc617839634f, State: Running, Role: LEADER
I20260811 22:17:23.734999 11565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120c5e38fe6b4ca29a7abc617839634f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41925 } }
I20260811 22:17:23.735286 11549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:23.737850 11566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "120c5e38fe6b4ca29a7abc617839634f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120c5e38fe6b4ca29a7abc617839634f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41925 } } }
I20260811 22:17:23.738219 11566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:23.740679 11567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 120c5e38fe6b4ca29a7abc617839634f. Latest consensus state: current_term: 1 leader_uuid: "120c5e38fe6b4ca29a7abc617839634f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "120c5e38fe6b4ca29a7abc617839634f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41925 } } }
I20260811 22:17:23.741112 11567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:23.742307 11573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:23.748095 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:23.726101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:56061 (local address 127.0.0.1:41925)
0811 22:17:23.726305 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:23.726312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:23.726706 (+ 394us) server_negotiation.cc:407] Connection header received
0811 22:17:23.726926 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:23.726931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:23.727021 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:23.727144 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:23.728835 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:23.730268 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:23.731620 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:23.732092 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:23.732242 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:23.735825 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:23.735834 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:23.735845 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:23.735894 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:23.741293 (+ 5399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:23.742603 (+ 1310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:23.742611 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:23.742617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:23.744632 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:23.745296 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:23.745301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:23.745304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:23.745758 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:23.746187 (+ 429us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:23.747278 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0811 22:17:23.747529 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:17:23.752871 11573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:23.764626 11573 catalog_manager.cc:1347] Generated new cluster ID: 62ee60a00ac3436eafb7836a141b7709
I20260811 22:17:23.764760 11573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:23.793538 11573 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:23.795097 11573 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:23.804556 11573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 120c5e38fe6b4ca29a7abc617839634f: Generated new TSK 0
I20260811 22:17:23.805627 11573 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:23.818849 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11442
I20260811 22:17:23.820969 11539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:23.916589 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11442
W20260811 22:17:23.949709 11423 connection.cc:535] client connection to 127.0.0.1:41925 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41925): Connection reset by peer (error 104)
2026-08-11T22:17:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:24 dist-test-slave-68xk krb5kdc[11603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:24 dist-test-slave-68xk krb5kdc[11603](info): set up 2 sockets
Aug 11 22:17:24 dist-test-slave-68xk krb5kdc[11603](info): commencing operation
krb5kdc: starting...
W20260811 22:17:25.277985 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:25 dist-test-slave-68xk krb5kdc[11603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:25.609732 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37765
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:25.962112 11618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:25.962507 11618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:25.962679 11618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:25.970043 11618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:25.970191 11618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:25.970312 11618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:25.970448 11618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:25.970530 11618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:25.980345 11618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:25.981732 11618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:25.983172 11618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:25.994772 11627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:25.994882 11628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:25.994917 11631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:25.996459 11618 server_base.cc:1034] running on GCE node
I20260811 22:17:25.997246 11618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:25.998582 11618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:25.999815 11618 hybrid_clock.cc:648] HybridClock initialized: now 1786486645999781 us; error 48 us; skew 500 ppm
I20260811 22:17:26.000162 11618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:25 dist-test-slave-68xk krb5kdc[11603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486645, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:26.003409 11618 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:26.005012 11618 webserver.cc:460] Webserver started at http://127.0.0.1:42669/ using document root <none> and password file <none>
I20260811 22:17:26.005702 11618 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:26.005859 11618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:26.006173 11618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:26.009305 11618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "c7bc9af6a0754157ab49700eee935ca7"
format_stamp: "Formatted at 2026-08-11 22:17:26 on dist-test-slave-68xk"
server_key: "67826616fad54dac9c06632d95b50b2c"
server_key_iv: "b02bad95373d0ab67bcd4d4e65a38f00"
server_key_version: "encryptionkey@0"
I20260811 22:17:26.010108 11618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "c7bc9af6a0754157ab49700eee935ca7"
format_stamp: "Formatted at 2026-08-11 22:17:26 on dist-test-slave-68xk"
server_key: "67826616fad54dac9c06632d95b50b2c"
server_key_iv: "b02bad95373d0ab67bcd4d4e65a38f00"
server_key_version: "encryptionkey@0"
I20260811 22:17:26.015588 11618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 22:17:26.019155 11646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:26.020517 11618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:17:26.020702 11618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "c7bc9af6a0754157ab49700eee935ca7"
format_stamp: "Formatted at 2026-08-11 22:17:26 on dist-test-slave-68xk"
server_key: "67826616fad54dac9c06632d95b50b2c"
server_key_iv: "b02bad95373d0ab67bcd4d4e65a38f00"
server_key_version: "encryptionkey@0"
I20260811 22:17:26.020906 11618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:26.036721 11618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:26.042140 11618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:26.042507 11618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:26.063426 11618 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37765
I20260811 22:17:26.063463 11724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37765 every 8 connection(s)
I20260811 22:17:26.064914 11618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:26.069022 11725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:26.072693 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11618
I20260811 22:17:26.072932 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:26.073514 10934 external_mini_cluster.cc:1428] Setting key 4da84c3cd0ff6786b62c4907bf9f2106
I20260811 22:17:26.082005 11725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7: Bootstrap starting.
I20260811 22:17:26.085767 11725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:26.087109 11725 log.cc:824] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:26.087313 11728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:26.075428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:44343 (local address 127.0.0.1:37765)
0811 22:17:26.076219 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:26.076231 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:26.076274 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:17:26.077935 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:26.077956 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:26.078355 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:26.078766 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:26.079366 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:26.080692 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:26.082662 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:26.083320 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:26.083449 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:26.084855 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:26.084862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:26.084874 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:26.085413 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:26.085483 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:26.085508 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:26.085629 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:26.086025 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 22:17:26.086208 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":366,"threads_started":1}
I20260811 22:17:26.091267 11725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7: No bootstrap required, opened a new log
I20260811 22:17:26.091442 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:26.100663 11725 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bc9af6a0754157ab49700eee935ca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37765 } }
I20260811 22:17:26.101174 11725 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:26.101310 11725 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7bc9af6a0754157ab49700eee935ca7, State: Initialized, Role: FOLLOWER
I20260811 22:17:26.102030 11725 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bc9af6a0754157ab49700eee935ca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37765 } }
I20260811 22:17:26.102362 11725 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:26.102555 11725 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:26.102779 11725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:26.103120 11728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:26.096041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:54797 (local address 127.0.0.1:37765)
0811 22:17:26.096246 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:26.096254 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:26.096598 (+ 344us) server_negotiation.cc:407] Connection header received
0811 22:17:26.096817 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:26.096821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:26.096885 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:26.096968 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:26.097588 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:26.098383 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:26.100357 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:26.100775 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:26.100864 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:26.102222 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:26.102227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:26.102231 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:26.102251 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:26.102320 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:26.102343 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:26.102507 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:26.102721 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 22:17:26.102795 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 22:17:26.105759 11725 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bc9af6a0754157ab49700eee935ca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37765 } }
I20260811 22:17:26.106274 11725 leader_election.cc:302] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7bc9af6a0754157ab49700eee935ca7; no voters:
I20260811 22:17:26.106916 11725 leader_election.cc:288] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:26.107050 11739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:26.108546 11739 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [term 1 LEADER]: Becoming Leader. State: Replica: c7bc9af6a0754157ab49700eee935ca7, State: Running, Role: LEADER
I20260811 22:17:26.109169 11725 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:26.109095 11739 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bc9af6a0754157ab49700eee935ca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37765 } }
I20260811 22:17:26.111842 11742 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7bc9af6a0754157ab49700eee935ca7. Latest consensus state: current_term: 1 leader_uuid: "c7bc9af6a0754157ab49700eee935ca7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bc9af6a0754157ab49700eee935ca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37765 } } }
I20260811 22:17:26.112167 11742 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:26.112444 11740 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7bc9af6a0754157ab49700eee935ca7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7bc9af6a0754157ab49700eee935ca7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37765 } } }
I20260811 22:17:26.112800 11740 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:26.114634 11753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:26.119983 11753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:26.128501 11753 catalog_manager.cc:1347] Generated new cluster ID: 75f8006a3e4a4ec487e60595f7aafad8
I20260811 22:17:26.128618 11753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:26.145959 11753 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:26.147373 11753 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:26.156567 11753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7bc9af6a0754157ab49700eee935ca7: Generated new TSK 0
I20260811 22:17:26.157642 11753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:26.167712 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11618
W20260811 22:17:26.169374 11718 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.210:44343
W20260811 22:17:26.170286 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11618: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:26.170476 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11618
W20260811 22:17:26.199889 11596 connection.cc:535] client connection to 127.0.0.1:37765 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37765): Connection reset by peer (error 104)
2026-08-11T22:17:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2249 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:26 dist-test-slave-68xk krb5kdc[11768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:26 dist-test-slave-68xk krb5kdc[11768](info): set up 2 sockets
Aug 11 22:17:26 dist-test-slave-68xk krb5kdc[11768](info): commencing operation
krb5kdc: starting...
W20260811 22:17:27.530040 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:27 dist-test-slave-68xk krb5kdc[11768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:27.865943 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38333
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:28.218149 11791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:28.218497 11791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:28.218717 11791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:28.226007 11791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:28.226110 11791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:28.226203 11791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:28.226315 11791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:28.226401 11791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:28.236555 11791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:28.237999 11791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:28.239424 11791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:28.252293 11802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:28.252332 11805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:28.252208 11803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:28.253500 11791 server_base.cc:1034] running on GCE node
I20260811 22:17:28.254509 11791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:28.256067 11791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:28.257296 11791 hybrid_clock.cc:648] HybridClock initialized: now 1786486648257247 us; error 49 us; skew 500 ppm
I20260811 22:17:28.257686 11791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:28 dist-test-slave-68xk krb5kdc[11768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486648, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:28.265770 11791 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:28.267596 11791 webserver.cc:460] Webserver started at http://127.0.0.1:34227/ using document root <none> and password file <none>
I20260811 22:17:28.268352 11791 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:28.268497 11791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:28.268834 11791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:28.272020 11791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "99d2b03a06624ed29a629744e6996991"
format_stamp: "Formatted at 2026-08-11 22:17:28 on dist-test-slave-68xk"
server_key: "54e48654ede7e622e7ea63a22995cacd"
server_key_iv: "5d7100d6522ed6de7fdcb84d4c91bdcd"
server_key_version: "encryptionkey@0"
I20260811 22:17:28.272830 11791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "99d2b03a06624ed29a629744e6996991"
format_stamp: "Formatted at 2026-08-11 22:17:28 on dist-test-slave-68xk"
server_key: "54e48654ede7e622e7ea63a22995cacd"
server_key_iv: "5d7100d6522ed6de7fdcb84d4c91bdcd"
server_key_version: "encryptionkey@0"
I20260811 22:17:28.278553 11791 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 22:17:28.282621 11815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:28.284237 11791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:17:28.284479 11791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "99d2b03a06624ed29a629744e6996991"
format_stamp: "Formatted at 2026-08-11 22:17:28 on dist-test-slave-68xk"
server_key: "54e48654ede7e622e7ea63a22995cacd"
server_key_iv: "5d7100d6522ed6de7fdcb84d4c91bdcd"
server_key_version: "encryptionkey@0"
I20260811 22:17:28.284709 11791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:28.291728 11791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:28.296829 11791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:28.297190 11791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:28.318545 11791 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38333
I20260811 22:17:28.318688 11895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38333 every 8 connection(s)
I20260811 22:17:28.320160 11791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:28.324153 11897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:28.329488 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11791
I20260811 22:17:28.329691 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:28.330173 10934 external_mini_cluster.cc:1428] Setting key 7eceac7ec7cdcc08cdc0498803bfe0e7
I20260811 22:17:28.336720 11897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991: Bootstrap starting.
I20260811 22:17:28.338593 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:28.332105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:47409 (local address 127.0.0.1:38333)
0811 22:17:28.333001 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:28.333013 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:28.333054 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:17:28.334954 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:28.334979 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:28.335316 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:28.335653 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:28.336843 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:28.336850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:28.336863 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:28.336906 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:28.336984 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:28.337011 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:28.337191 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:28.337376 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 22:17:28.337594 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":438,"threads_started":1}
I20260811 22:17:28.340847 11897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:28.342170 11897 log.cc:824] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:28.342571 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:28.345835 11897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991: No bootstrap required, opened a new log
I20260811 22:17:28.351498 11901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:28.348538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:32917 (local address 127.0.0.1:38333)
0811 22:17:28.348723 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:28.348728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:28.349143 (+ 415us) server_negotiation.cc:407] Connection header received
0811 22:17:28.349395 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:28.349403 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:28.349466 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:28.349570 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:28.350711 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:28.350716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:28.350720 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:28.350741 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:28.350795 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:28.350812 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:28.350925 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:28.351095 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:17:28.351164 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:17:28.356216 11897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d2b03a06624ed29a629744e6996991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } }
I20260811 22:17:28.356654 11897 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:28.356779 11897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d2b03a06624ed29a629744e6996991, State: Initialized, Role: FOLLOWER
I20260811 22:17:28.357388 11897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d2b03a06624ed29a629744e6996991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } }
I20260811 22:17:28.357677 11897 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:28.357851 11897 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:28.358084 11897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:28.360677 11897 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d2b03a06624ed29a629744e6996991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } }
I20260811 22:17:28.361173 11897 leader_election.cc:302] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99d2b03a06624ed29a629744e6996991; no voters:
I20260811 22:17:28.361721 11897 leader_election.cc:288] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:28.361883 11909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:28.362703 11909 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [term 1 LEADER]: Becoming Leader. State: Replica: 99d2b03a06624ed29a629744e6996991, State: Running, Role: LEADER
I20260811 22:17:28.363332 11909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d2b03a06624ed29a629744e6996991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } }
I20260811 22:17:28.364127 11897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:28.366297 11913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99d2b03a06624ed29a629744e6996991. Latest consensus state: current_term: 1 leader_uuid: "99d2b03a06624ed29a629744e6996991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d2b03a06624ed29a629744e6996991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } } }
I20260811 22:17:28.366943 11913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:28.367560 11911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99d2b03a06624ed29a629744e6996991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d2b03a06624ed29a629744e6996991" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38333 } } }
I20260811 22:17:28.367926 11911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:28.369648 11919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:28.374702 11919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:28.384894 11919 catalog_manager.cc:1347] Generated new cluster ID: b7c846b6c3804849b609afd3200f3d34
I20260811 22:17:28.385015 11919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:28.414147 11919 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:28.414671 11919 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:28.429169 11919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99d2b03a06624ed29a629744e6996991: Generated new TSK 0
I20260811 22:17:28.430294 11919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:28.446758 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11791
W20260811 22:17:28.447979 11888 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.210:47409
W20260811 22:17:28.448819 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11791: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:28.449007 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11791
2026-08-11T22:17:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2277 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:28 dist-test-slave-68xk krb5kdc[11944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:28 dist-test-slave-68xk krb5kdc[11944](info): set up 2 sockets
Aug 11 22:17:28 dist-test-slave-68xk krb5kdc[11944](info): commencing operation
krb5kdc: starting...
W20260811 22:17:29.804885 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:30 dist-test-slave-68xk krb5kdc[11944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:30.137758 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:30.481284 11961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:30.481657 11961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:30.481861 11961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:30.489166 11961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:30.489341 11961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:30.489431 11961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:30.489524 11961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:30.489606 11961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:30.499495 11961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45609
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:30.500854 11961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:30.502178 11961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:30.514698 11966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:30.514724 11973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:30.514639 11969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:30.515452 11961 server_base.cc:1034] running on GCE node
I20260811 22:17:30.516347 11961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:30.517869 11961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:30.519116 11961 hybrid_clock.cc:648] HybridClock initialized: now 1786486650519077 us; error 65 us; skew 500 ppm
I20260811 22:17:30.519495 11961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:30 dist-test-slave-68xk krb5kdc[11944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486650, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:30.522735 11961 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:30.524163 11961 webserver.cc:460] Webserver started at http://127.0.0.1:38671/ using document root <none> and password file <none>
I20260811 22:17:30.524788 11961 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:30.524888 11961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:30.525156 11961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:30.528165 11961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "6cb3289827a34fd5bd03705c7904cf51"
format_stamp: "Formatted at 2026-08-11 22:17:30 on dist-test-slave-68xk"
server_key: "4fcd1c632c407d8870d7f057b2f1acf3"
server_key_iv: "bfbfa3fc4796a8f594f100c6783a9f03"
server_key_version: "encryptionkey@0"
I20260811 22:17:30.528910 11961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "6cb3289827a34fd5bd03705c7904cf51"
format_stamp: "Formatted at 2026-08-11 22:17:30 on dist-test-slave-68xk"
server_key: "4fcd1c632c407d8870d7f057b2f1acf3"
server_key_iv: "bfbfa3fc4796a8f594f100c6783a9f03"
server_key_version: "encryptionkey@0"
I20260811 22:17:30.533954 11961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:17:30.537604 11987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:30.539011 11961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:17:30.539213 11961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "6cb3289827a34fd5bd03705c7904cf51"
format_stamp: "Formatted at 2026-08-11 22:17:30 on dist-test-slave-68xk"
server_key: "4fcd1c632c407d8870d7f057b2f1acf3"
server_key_iv: "bfbfa3fc4796a8f594f100c6783a9f03"
server_key_version: "encryptionkey@0"
I20260811 22:17:30.539405 11961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:30.558241 11961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:30.562610 11961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:30.562930 11961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:30.582016 11961 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45609
I20260811 22:17:30.582050 12068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45609 every 8 connection(s)
I20260811 22:17:30.583534 11961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:30.587744 12069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:30.588666 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 11961
I20260811 22:17:30.588881 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:30.589464 10934 external_mini_cluster.cc:1428] Setting key 65e73649066a57a25afdda7d98db86d9
I20260811 22:17:30.602660 12069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51: Bootstrap starting.
Aug 11 22:17:30 dist-test-slave-68xk krb5kdc[11944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486650, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:17:30.606745 12069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:30.607905 12069 log.cc:824] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:30.611016 12069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51: No bootstrap required, opened a new log
I20260811 22:17:30.614753 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:30.591494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:60985 (local address 127.0.0.1:45609)
0811 22:17:30.592426 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:30.592439 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:30.592489 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:17:30.594363 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:30.594392 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:30.594927 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:30.595316 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:30.597348 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:30.598918 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:30.600883 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:30.601594 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:30.601750 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:30.607038 (+ 5288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:30.607048 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:30.607064 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:30.607108 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:30.610361 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:30.611230 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:30.611241 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:30.611254 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:30.611413 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:30.611915 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:30.611920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:30.611922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:30.612449 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:30.612831 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:30.613322 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 22:17:30.613666 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":406,"threads_started":1}
I20260811 22:17:30.619040 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:30.620182 12069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb3289827a34fd5bd03705c7904cf51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45609 } }
I20260811 22:17:30.620553 12069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:30.620663 12069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb3289827a34fd5bd03705c7904cf51, State: Initialized, Role: FOLLOWER
I20260811 22:17:30.621224 12069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb3289827a34fd5bd03705c7904cf51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45609 } }
I20260811 22:17:30.621483 12069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:30.621672 12069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:30.621857 12069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:30.624501 12069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb3289827a34fd5bd03705c7904cf51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45609 } }
I20260811 22:17:30.625151 12069 leader_election.cc:302] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cb3289827a34fd5bd03705c7904cf51; no voters:
I20260811 22:17:30.625840 12069 leader_election.cc:288] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:30.625989 12085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:30.628015 12069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:30.628021 12085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [term 1 LEADER]: Becoming Leader. State: Replica: 6cb3289827a34fd5bd03705c7904cf51, State: Running, Role: LEADER
I20260811 22:17:30.628684 12085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb3289827a34fd5bd03705c7904cf51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45609 } }
I20260811 22:17:30.631836 12087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cb3289827a34fd5bd03705c7904cf51. Latest consensus state: current_term: 1 leader_uuid: "6cb3289827a34fd5bd03705c7904cf51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb3289827a34fd5bd03705c7904cf51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45609 } } }
I20260811 22:17:30.632447 12086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cb3289827a34fd5bd03705c7904cf51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb3289827a34fd5bd03705c7904cf51" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45609 } } }
I20260811 22:17:30.633303 12086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:30.633148 12087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:30.640466 12093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:30.644532 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:30.624252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:39399 (local address 127.0.0.1:45609)
0811 22:17:30.624451 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:30.624457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:30.624995 (+ 538us) server_negotiation.cc:407] Connection header received
0811 22:17:30.625197 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:30.625203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:30.625297 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:30.625417 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:30.627723 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:30.628994 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:30.630803 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:30.631557 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:30.631711 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:30.635630 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:30.635641 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:30.635649 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:30.635713 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:30.640350 (+ 4637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:30.641595 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:30.641601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:30.641604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:30.641783 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:30.642439 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:30.642444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:30.642448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:30.643052 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:30.643425 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:30.643966 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 22:17:30.644149 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:17:30.647207 12093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:30.657085 12093 catalog_manager.cc:1347] Generated new cluster ID: 53437177ba584e52a272a90ff6fca9b6
I20260811 22:17:30.657218 12093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:30.677412 12093 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:30.678898 12093 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:30.685395 12093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cb3289827a34fd5bd03705c7904cf51: Generated new TSK 0
I20260811 22:17:30.686551 12093 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:30.699786 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 11961
I20260811 22:17:30.701588 12061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:30.794360 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 11961
W20260811 22:17:30.817963 11941 connection.cc:535] client connection to 127.0.0.1:45609 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45609): Connection reset by peer (error 104)
2026-08-11T22:17:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:30 dist-test-slave-68xk krb5kdc[12114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:30 dist-test-slave-68xk krb5kdc[12114](info): set up 2 sockets
Aug 11 22:17:30 dist-test-slave-68xk krb5kdc[12114](info): commencing operation
krb5kdc: starting...
W20260811 22:17:32.180280 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:32 dist-test-slave-68xk krb5kdc[12114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:32.518877 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:32.872282 12137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:32.872579 12137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:32.872674 12137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:32.879962 12137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:32.880077 12137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:32.880127 12137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:32.880184 12137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:32.880236 12137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:32.890116 12137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35739
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:32.891525 12137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:32.892805 12137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:32.905668 12145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:32.905773 12147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:32.905799 12150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:32.907048 12137 server_base.cc:1034] running on GCE node
I20260811 22:17:32.907872 12137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:32.909268 12137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:32.910507 12137 hybrid_clock.cc:648] HybridClock initialized: now 1786486652910462 us; error 42 us; skew 500 ppm
I20260811 22:17:32.910815 12137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:32 dist-test-slave-68xk krb5kdc[12114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486652, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:32.914058 12137 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:32.915694 12137 webserver.cc:460] Webserver started at http://127.0.0.1:43017/ using document root <none> and password file <none>
I20260811 22:17:32.916419 12137 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:32.916581 12137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:32.916915 12137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:32.920037 12137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "adfaea7ed5544443867f845a0446fe4f"
format_stamp: "Formatted at 2026-08-11 22:17:32 on dist-test-slave-68xk"
server_key: "867db2c341380f811ca5a97aff8c45b7"
server_key_iv: "8b46111d37647d7fd3ebfbe4e2d15293"
server_key_version: "encryptionkey@0"
I20260811 22:17:32.920848 12137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "adfaea7ed5544443867f845a0446fe4f"
format_stamp: "Formatted at 2026-08-11 22:17:32 on dist-test-slave-68xk"
server_key: "867db2c341380f811ca5a97aff8c45b7"
server_key_iv: "8b46111d37647d7fd3ebfbe4e2d15293"
server_key_version: "encryptionkey@0"
I20260811 22:17:32.926115 12137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 22:17:32.929745 12161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:32.931151 12137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:17:32.931352 12137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "adfaea7ed5544443867f845a0446fe4f"
format_stamp: "Formatted at 2026-08-11 22:17:32 on dist-test-slave-68xk"
server_key: "867db2c341380f811ca5a97aff8c45b7"
server_key_iv: "8b46111d37647d7fd3ebfbe4e2d15293"
server_key_version: "encryptionkey@0"
I20260811 22:17:32.931576 12137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:32.957340 12137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:32.962167 12137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:32.962603 12137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:32.982394 12137 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35739
I20260811 22:17:32.982493 12247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35739 every 8 connection(s)
I20260811 22:17:32.983963 12137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:32.987900 12250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:32.987989 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 12137
I20260811 22:17:32.988186 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:32.988626 10934 external_mini_cluster.cc:1428] Setting key ac5798e96b1225ab368f8350d5a66f9d
I20260811 22:17:33.002197 12250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f: Bootstrap starting.
I20260811 22:17:33.002527 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:32.990476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:42165 (local address 127.0.0.1:35739)
0811 22:17:32.991372 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:32.991383 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:32.991432 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:17:32.992966 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:32.992989 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:32.993402 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:32.993744 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:32.994492 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:32.995852 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:32.997753 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:32.998512 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:32.998658 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:33.000037 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:33.000048 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:33.000062 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:33.000615 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:33.000689 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:33.000713 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:33.000831 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:33.001236 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 22:17:33.001436 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":435,"threads_started":1}
I20260811 22:17:33.006465 12250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:33.007200 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:33.007695 12250 log.cc:824] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:33.011547 12250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f: No bootstrap required, opened a new log
I20260811 22:17:33.021881 12254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:33.015506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:38701 (local address 127.0.0.1:35739)
0811 22:17:33.015698 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:33.015704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:33.015724 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:17:33.015780 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:33.015784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:33.015864 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:33.015969 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:33.016505 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:33.017676 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:33.019097 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:33.019813 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:33.019960 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:33.021099 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:33.021106 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:33.021111 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:33.021136 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:33.021198 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:33.021226 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:33.021338 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:33.021518 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 22:17:33.021571 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:17:33.022248 12250 raft_consensus.cc:348] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfaea7ed5544443867f845a0446fe4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } }
I20260811 22:17:33.022770 12250 raft_consensus.cc:374] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:33.022904 12250 raft_consensus.cc:729] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adfaea7ed5544443867f845a0446fe4f, State: Initialized, Role: FOLLOWER
I20260811 22:17:33.023614 12250 consensus_queue.cc:260] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfaea7ed5544443867f845a0446fe4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } }
I20260811 22:17:33.023921 12250 raft_consensus.cc:388] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:33.024036 12250 raft_consensus.cc:482] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:33.024183 12250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:33.027026 12250 raft_consensus.cc:504] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfaea7ed5544443867f845a0446fe4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } }
I20260811 22:17:33.027563 12250 leader_election.cc:302] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adfaea7ed5544443867f845a0446fe4f; no voters:
I20260811 22:17:33.028268 12250 leader_election.cc:288] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:33.028414 12269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:33.031056 12250 sys_catalog.cc:564] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:33.031404 12269 raft_consensus.cc:686] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [term 1 LEADER]: Becoming Leader. State: Replica: adfaea7ed5544443867f845a0446fe4f, State: Running, Role: LEADER
I20260811 22:17:33.031940 12269 consensus_queue.cc:237] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfaea7ed5544443867f845a0446fe4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } }
I20260811 22:17:33.036113 12270 sys_catalog.cc:455] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adfaea7ed5544443867f845a0446fe4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfaea7ed5544443867f845a0446fe4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } } }
I20260811 22:17:33.036597 12270 sys_catalog.cc:458] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:33.037134 12272 sys_catalog.cc:455] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader adfaea7ed5544443867f845a0446fe4f. Latest consensus state: current_term: 1 leader_uuid: "adfaea7ed5544443867f845a0446fe4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adfaea7ed5544443867f845a0446fe4f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35739 } } }
I20260811 22:17:33.037498 12272 sys_catalog.cc:458] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:33.043555 12280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:33.048843 12280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:33.056784 12280 catalog_manager.cc:1347] Generated new cluster ID: 8e84c84b8d004824a024fe172cab66c8
I20260811 22:17:33.056897 12280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:33.091311 12280 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:33.093243 12280 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:33.110673 12280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adfaea7ed5544443867f845a0446fe4f: Generated new TSK 0
I20260811 22:17:33.111673 12280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:33.120428 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12137
W20260811 22:17:33.121654 12241 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.210:42165
W20260811 22:17:33.122602 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12137: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:33.122773 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 12137
W20260811 22:17:33.150738 12112 connection.cc:535] client connection to 127.0.0.1:35739 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35739): Connection reset by peer (error 104)
2026-08-11T22:17:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:33 dist-test-slave-68xk krb5kdc[12296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:33 dist-test-slave-68xk krb5kdc[12296](info): set up 2 sockets
Aug 11 22:17:33 dist-test-slave-68xk krb5kdc[12296](info): commencing operation
krb5kdc: starting...
W20260811 22:17:34.482069 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:34 dist-test-slave-68xk krb5kdc[12296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:17:34.828536 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36173
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:35.185387 12312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:35.185662 12312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:35.185828 12312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:35.193120 12312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:35.193238 12312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:35.193374 12312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:35.193475 12312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:35.193542 12312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:35.203553 12312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36173
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:35.204870 12312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:35.206211 12312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:35.218290 12325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:35.218268 12319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:35.218555 12322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:35.220023 12312 server_base.cc:1034] running on GCE node
I20260811 22:17:35.220710 12312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:35.222021 12312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:35.223237 12312 hybrid_clock.cc:648] HybridClock initialized: now 1786486655223187 us; error 32 us; skew 500 ppm
I20260811 22:17:35.223591 12312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:35 dist-test-slave-68xk krb5kdc[12296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486655, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:35.233098 12312 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:17:35.234679 12312 webserver.cc:460] Webserver started at http://127.0.0.1:34603/ using document root <none> and password file <none>
I20260811 22:17:35.235385 12312 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:35.235499 12312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:35.235782 12312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:35.239228 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d"
format_stamp: "Formatted at 2026-08-11 22:17:35 on dist-test-slave-68xk"
server_key: "87ba30caf4b33faacbaa8ad9d7cab820"
server_key_iv: "c1000d10ac3ae3aff9226f21677054fd"
server_key_version: "encryptionkey@0"
I20260811 22:17:35.240033 12312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d"
format_stamp: "Formatted at 2026-08-11 22:17:35 on dist-test-slave-68xk"
server_key: "87ba30caf4b33faacbaa8ad9d7cab820"
server_key_iv: "c1000d10ac3ae3aff9226f21677054fd"
server_key_version: "encryptionkey@0"
I20260811 22:17:35.245766 12312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:17:35.249506 12334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:35.251057 12312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:17:35.251291 12312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d"
format_stamp: "Formatted at 2026-08-11 22:17:35 on dist-test-slave-68xk"
server_key: "87ba30caf4b33faacbaa8ad9d7cab820"
server_key_iv: "c1000d10ac3ae3aff9226f21677054fd"
server_key_version: "encryptionkey@0"
I20260811 22:17:35.251506 12312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:35.257887 12312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:35.262373 12312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:35.262713 12312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:35.282125 12312 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36173
I20260811 22:17:35.282173 12419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36173 every 8 connection(s)
I20260811 22:17:35.283519 12312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:35.287232 12420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:35.291940 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 12312
I20260811 22:17:35.292124 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:35.292663 10934 external_mini_cluster.cc:1428] Setting key ad901ae0de991580e180a0f3fde0920a
I20260811 22:17:35.298661 12420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d: Bootstrap starting.
I20260811 22:17:35.300710 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:35.294580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:49393 (local address 127.0.0.1:36173)
0811 22:17:35.295340 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:35.295354 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:35.295394 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:17:35.297380 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:35.297404 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:35.297725 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:35.298102 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:35.299183 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:35.299189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:35.299202 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:35.299216 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:35.299275 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:35.299296 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:35.299532 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:35.299662 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 22:17:35.299848 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":330,"threads_started":1}
I20260811 22:17:35.302027 12420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:35.303435 12420 log.cc:824] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:35.304868 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:35.306280 12420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d: No bootstrap required, opened a new log
I20260811 22:17:35.312424 12423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:35.309808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.210:45527 (local address 127.0.0.1:36173)
0811 22:17:35.310012 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:35.310017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:35.310324 (+ 307us) server_negotiation.cc:407] Connection header received
0811 22:17:35.310581 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:35.310585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:35.310639 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:35.310754 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:35.311746 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:35.311756 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:35.311759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:35.311774 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:35.311815 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:35.311830 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:35.311966 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:35.312082 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 22:17:35.312155 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:17:35.315222 12420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260811 22:17:35.315609 12420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:35.315747 12420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee8ff5d891f4302a2a5b83bbeb6317d, State: Initialized, Role: FOLLOWER
I20260811 22:17:35.316354 12420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260811 22:17:35.316617 12420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:35.316776 12420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:35.316958 12420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:35.319393 12420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260811 22:17:35.319907 12420 leader_election.cc:302] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ee8ff5d891f4302a2a5b83bbeb6317d; no voters:
I20260811 22:17:35.320492 12420 leader_election.cc:288] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:35.320649 12436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:35.321384 12436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [term 1 LEADER]: Becoming Leader. State: Replica: 6ee8ff5d891f4302a2a5b83bbeb6317d, State: Running, Role: LEADER
I20260811 22:17:35.321957 12436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } }
I20260811 22:17:35.322688 12420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:35.325042 12441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ee8ff5d891f4302a2a5b83bbeb6317d. Latest consensus state: current_term: 1 leader_uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } } }
I20260811 22:17:35.325547 12441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:35.330529 12440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ee8ff5d891f4302a2a5b83bbeb6317d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36173 } } }
I20260811 22:17:35.330857 12449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:35.330945 12440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:35.335927 12449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:35.344882 12449 catalog_manager.cc:1347] Generated new cluster ID: 770286d24cb54e208d8864527dc9db4c
I20260811 22:17:35.344997 12449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:35.388098 12449 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:35.388675 12449 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:35.398938 12449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ee8ff5d891f4302a2a5b83bbeb6317d: Generated new TSK 0
I20260811 22:17:35.400121 12449 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:35.421315 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12312
W20260811 22:17:35.423343 12413 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.210:49393
W20260811 22:17:35.424129 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12312: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:35.424283 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 12312
2026-08-11T22:17:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2299 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:35 dist-test-slave-68xk krb5kdc[12465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:35 dist-test-slave-68xk krb5kdc[12465](info): set up 2 sockets
Aug 11 22:17:35 dist-test-slave-68xk krb5kdc[12465](info): commencing operation
krb5kdc: starting...
W20260811 22:17:36.880570 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.411s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:37 dist-test-slave-68xk krb5kdc[12465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:17:37.230206 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:45491
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:40203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:45491
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:37.585558 12484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:37.585928 12484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:37.586133 12484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:37.594465 12484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:37.594650 12484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:37.594748 12484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:37.594892 12484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:37.594972 12484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:37.605528 12484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:40203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:45491
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:45491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:37.607054 12484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:37.608475 12484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:37.621142 12496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:37.621186 12494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:37.621218 12493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:37.621959 12484 server_base.cc:1034] running on GCE node
I20260811 22:17:37.622802 12484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:37.624217 12484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:37.625488 12484 hybrid_clock.cc:648] HybridClock initialized: now 1786486657625441 us; error 58 us; skew 500 ppm
I20260811 22:17:37.625896 12484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:37 dist-test-slave-68xk krb5kdc[12465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486657, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:37.629227 12484 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:17:37.631028 12484 webserver.cc:460] Webserver started at http://127.10.173.190:35163/ using document root <none> and password file <none>
I20260811 22:17:37.631826 12484 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:37.631990 12484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:37.632373 12484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:37.635717 12484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "819ef31a06c548798fa7a1afea96ee2c"
format_stamp: "Formatted at 2026-08-11 22:17:37 on dist-test-slave-68xk"
server_key: "d28b51ab8f19238da134e7d79fa0d3b7"
server_key_iv: "994fc4fb9a1d09f842ca82ccbd8fdcb5"
server_key_version: "encryptionkey@0"
I20260811 22:17:37.636629 12484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "819ef31a06c548798fa7a1afea96ee2c"
format_stamp: "Formatted at 2026-08-11 22:17:37 on dist-test-slave-68xk"
server_key: "d28b51ab8f19238da134e7d79fa0d3b7"
server_key_iv: "994fc4fb9a1d09f842ca82ccbd8fdcb5"
server_key_version: "encryptionkey@0"
I20260811 22:17:37.642443 12484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:17:37.646517 12507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:37.647855 12484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:17:37.648064 12484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "819ef31a06c548798fa7a1afea96ee2c"
format_stamp: "Formatted at 2026-08-11 22:17:37 on dist-test-slave-68xk"
server_key: "d28b51ab8f19238da134e7d79fa0d3b7"
server_key_iv: "994fc4fb9a1d09f842ca82ccbd8fdcb5"
server_key_version: "encryptionkey@0"
I20260811 22:17:37.648299 12484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:37.668620 12484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:37.673018 12484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:37.673359 12484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:37.693789 12484 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:45491
I20260811 22:17:37.693809 12590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:45491 every 8 connection(s)
I20260811 22:17:37.695240 12484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:37.699110 12592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:37.703444 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 12484
I20260811 22:17:37.703639 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:37.704214 10934 external_mini_cluster.cc:1428] Setting key f8a17b81a53309a78b1ecdfdb58af99d
I20260811 22:17:37.712697 12592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c: Bootstrap starting.
I20260811 22:17:37.716384 12592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:37.717540 12592 log.cc:824] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:17:37 dist-test-slave-68xk krb5kdc[12465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:17:37.721086 12592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c: No bootstrap required, opened a new log
I20260811 22:17:37.728310 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:37.706238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38509 (local address 127.10.173.190:45491)
0811 22:17:37.707125 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:37.707137 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:37.707179 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:17:37.708871 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:37.708895 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:37.709299 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:37.709724 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:37.711402 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:37.712752 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:37.714097 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:37.714869 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:37.715072 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:37.720060 (+ 4988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:37.720068 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:37.720083 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:37.720127 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:37.723515 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:37.724567 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:37.724574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:37.724580 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:37.724759 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:37.725353 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:37.725356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:37.725358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:37.725898 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:37.726138 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:37.726878 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 22:17:37.727231 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":452,"threads_started":1}
I20260811 22:17:37.731140 12592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819ef31a06c548798fa7a1afea96ee2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45491 } }
I20260811 22:17:37.731606 12592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:37.731734 12592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 819ef31a06c548798fa7a1afea96ee2c, State: Initialized, Role: FOLLOWER
I20260811 22:17:37.732451 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:37.732350 12592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819ef31a06c548798fa7a1afea96ee2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45491 } }
I20260811 22:17:37.732676 12592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:37.732815 12592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:37.732986 12592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:37.736683 12592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819ef31a06c548798fa7a1afea96ee2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45491 } }
I20260811 22:17:37.737390 12592 leader_election.cc:302] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 819ef31a06c548798fa7a1afea96ee2c; no voters:
I20260811 22:17:37.738081 12592 leader_election.cc:288] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:37.738189 12608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:37.739440 12608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [term 1 LEADER]: Becoming Leader. State: Replica: 819ef31a06c548798fa7a1afea96ee2c, State: Running, Role: LEADER
I20260811 22:17:37.740243 12608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819ef31a06c548798fa7a1afea96ee2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45491 } }
I20260811 22:17:37.743192 12592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:37.743618 12611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "819ef31a06c548798fa7a1afea96ee2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819ef31a06c548798fa7a1afea96ee2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45491 } } }
I20260811 22:17:37.743647 12610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 819ef31a06c548798fa7a1afea96ee2c. Latest consensus state: current_term: 1 leader_uuid: "819ef31a06c548798fa7a1afea96ee2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "819ef31a06c548798fa7a1afea96ee2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 45491 } } }
I20260811 22:17:37.744163 12610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:37.744168 12611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:37.749116 12620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:37.760125 12620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:37.761070 12599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:37.738859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54945 (local address 127.10.173.190:45491)
0811 22:17:37.739121 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:37.739131 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:37.740677 (+ 1546us) server_negotiation.cc:407] Connection header received
0811 22:17:37.740921 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:37.740931 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:37.741010 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:37.741110 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:37.743363 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:37.744772 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:37.746874 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:37.747766 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:37.747938 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:37.752319 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:37.752328 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:37.752336 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:37.752393 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:37.757047 (+ 4654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:37.758098 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:37.758109 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:37.758116 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:37.758327 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:37.759066 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:37.759071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:37.759075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:37.759628 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:37.759987 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:37.760398 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 22:17:37.760620 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:17:37.770828 12620 catalog_manager.cc:1347] Generated new cluster ID: c2c50e1e98444d74abe306b7cc75041c
I20260811 22:17:37.770946 12620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:37.793434 12620 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:37.794863 12620 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:37.806593 12620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 819ef31a06c548798fa7a1afea96ee2c: Generated new TSK 0
I20260811 22:17:37.807433 12620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:37.818816 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12484
I20260811 22:17:37.820235 12585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:37.914860 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 12484
W20260811 22:17:37.942476 12461 connection.cc:535] client connection to 127.10.173.190:45491 recv error: Network error: failed to read from TLS socket (remote: 127.10.173.190:45491): Connection reset by peer (error 104)
2026-08-11T22:17:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:38 dist-test-slave-68xk krb5kdc[12644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:38 dist-test-slave-68xk krb5kdc[12644](info): set up 2 sockets
Aug 11 22:17:38 dist-test-slave-68xk krb5kdc[12644](info): commencing operation
krb5kdc: starting...
W20260811 22:17:39.284186 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.323s user 0.006s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:39 dist-test-slave-68xk krb5kdc[12644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:17:39.605911 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38557
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:42913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:38557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:39.962221 12660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:39.962656 12660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:39.962832 12660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:39.970538 12660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:39.970690 12660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:39.970796 12660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:39.970906 12660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:39.970989 12660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:39.981590 12660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:42913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:38557
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:38557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:39.983000 12660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:39.984357 12660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:39.996354 12667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:39.996503 12671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:39.996503 12666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:39.997516 12660 server_base.cc:1034] running on GCE node
I20260811 22:17:39.998296 12660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:39.999754 12660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:40.001010 12660 hybrid_clock.cc:648] HybridClock initialized: now 1786486660000957 us; error 44 us; skew 500 ppm
I20260811 22:17:40.001538 12660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:40 dist-test-slave-68xk krb5kdc[12644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486660, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:40.005123 12660 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:17:40.006903 12660 webserver.cc:460] Webserver started at http://127.10.173.190:45909/ using document root <none> and password file <none>
I20260811 22:17:40.007697 12660 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:40.007849 12660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:40.008193 12660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:40.011605 12660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "3b94176b237d4c72a2e4ac3c8ad848c9"
format_stamp: "Formatted at 2026-08-11 22:17:40 on dist-test-slave-68xk"
server_key: "2e0fefdde826e2b0ab632fa9a211eb89"
server_key_iv: "2cc3a0b8f88ea393bf261285769216d7"
server_key_version: "encryptionkey@0"
I20260811 22:17:40.012498 12660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "3b94176b237d4c72a2e4ac3c8ad848c9"
format_stamp: "Formatted at 2026-08-11 22:17:40 on dist-test-slave-68xk"
server_key: "2e0fefdde826e2b0ab632fa9a211eb89"
server_key_iv: "2cc3a0b8f88ea393bf261285769216d7"
server_key_version: "encryptionkey@0"
I20260811 22:17:40.018546 12660 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 22:17:40.022766 12679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:40.024199 12660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 22:17:40.024416 12660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "3b94176b237d4c72a2e4ac3c8ad848c9"
format_stamp: "Formatted at 2026-08-11 22:17:40 on dist-test-slave-68xk"
server_key: "2e0fefdde826e2b0ab632fa9a211eb89"
server_key_iv: "2cc3a0b8f88ea393bf261285769216d7"
server_key_version: "encryptionkey@0"
I20260811 22:17:40.024644 12660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:40.044109 12660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:40.048683 12660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:40.048985 12660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:40.068648 12660 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:38557
I20260811 22:17:40.068673 12767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:38557 every 8 connection(s)
I20260811 22:17:40.070164 12660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:40.074108 12768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:40.079819 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 12660
I20260811 22:17:40.080037 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:40.080490 10934 external_mini_cluster.cc:1428] Setting key 0425c5f7c20cc89a81490583883bc1a3
I20260811 22:17:40.085330 12768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9: Bootstrap starting.
I20260811 22:17:40.088555 12768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:40.089668 12768 log.cc:824] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:40.093065 12768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9: No bootstrap required, opened a new log
I20260811 22:17:40.094520 12775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:40.082348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42297 (local address 127.10.173.190:38557)
0811 22:17:40.083288 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:40.083304 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:40.083345 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:17:40.085077 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:40.085106 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:40.085541 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:40.085939 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:40.086510 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:40.087837 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:40.089292 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:40.090070 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:40.090313 (+ 243us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:40.091692 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:40.091702 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:40.091721 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:40.092442 (+ 721us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:40.092540 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:40.092574 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:40.092757 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:40.093003 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 22:17:40.093182 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":392,"threads_started":1}
I20260811 22:17:40.098987 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:40.103786 12768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b94176b237d4c72a2e4ac3c8ad848c9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38557 } }
I20260811 22:17:40.104256 12768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:40.104434 12768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b94176b237d4c72a2e4ac3c8ad848c9, State: Initialized, Role: FOLLOWER
I20260811 22:17:40.105109 12768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b94176b237d4c72a2e4ac3c8ad848c9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38557 } }
I20260811 22:17:40.105445 12768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:40.105618 12768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:40.105830 12768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:40.108757 12768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b94176b237d4c72a2e4ac3c8ad848c9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38557 } }
I20260811 22:17:40.109491 12768 leader_election.cc:302] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b94176b237d4c72a2e4ac3c8ad848c9; no voters:
I20260811 22:17:40.110198 12768 leader_election.cc:288] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:40.110358 12792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:40.112505 12775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:40.104561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43683 (local address 127.10.173.190:38557)
0811 22:17:40.104858 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:40.104864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:40.105210 (+ 346us) server_negotiation.cc:407] Connection header received
0811 22:17:40.105382 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:40.105387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:40.105477 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:40.105612 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:40.106243 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:40.107447 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:40.108893 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:40.109576 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:40.109718 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:40.111423 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:40.111442 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:40.111451 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:40.111495 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:40.111562 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:40.111592 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:40.111729 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:40.112076 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 22:17:40.112138 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:17:40.112829 12768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:40.112780 12792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [term 1 LEADER]: Becoming Leader. State: Replica: 3b94176b237d4c72a2e4ac3c8ad848c9, State: Running, Role: LEADER
I20260811 22:17:40.113917 12792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b94176b237d4c72a2e4ac3c8ad848c9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38557 } }
I20260811 22:17:40.117066 12793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b94176b237d4c72a2e4ac3c8ad848c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b94176b237d4c72a2e4ac3c8ad848c9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38557 } } }
I20260811 22:17:40.117530 12793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:40.118191 12796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b94176b237d4c72a2e4ac3c8ad848c9. Latest consensus state: current_term: 1 leader_uuid: "3b94176b237d4c72a2e4ac3c8ad848c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b94176b237d4c72a2e4ac3c8ad848c9" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 38557 } } }
I20260811 22:17:40.118549 12796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:40.122933 12809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:40.127504 12809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:40.135696 12809 catalog_manager.cc:1347] Generated new cluster ID: 0696df3973454be6a632c88dffe5ac12
I20260811 22:17:40.135829 12809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:40.142505 12809 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:40.144284 12809 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:40.151692 12809 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b94176b237d4c72a2e4ac3c8ad848c9: Generated new TSK 0
I20260811 22:17:40.152765 12809 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:40.167052 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12660
W20260811 22:17:40.168287 12759 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42297
W20260811 22:17:40.169080 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12660: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:40.169236 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 12660
W20260811 22:17:40.196585 12640 connection.cc:535] client connection to 127.10.173.190:38557 recv error: Network error: failed to read from TLS socket (remote: 127.10.173.190:38557): Connection reset by peer (error 104)
2026-08-11T22:17:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2253 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:40 dist-test-slave-68xk krb5kdc[12826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:40 dist-test-slave-68xk krb5kdc[12826](info): set up 2 sockets
Aug 11 22:17:40 dist-test-slave-68xk krb5kdc[12826](info): commencing operation
krb5kdc: starting...
W20260811 22:17:41.571247 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:41 dist-test-slave-68xk krb5kdc[12826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:17:41.902338 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41545
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:41545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:42.279031 12846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:42.279342 12846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:42.279445 12846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:42.286545 12846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:42.286655 12846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:42.286708 12846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:42.286770 12846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:42.286814 12846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:42.296669 12846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:41545
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:41545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:42.297953 12846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:42.299270 12846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:42.311292 12852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:42.311224 12857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:42.311153 12854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:42.312444 12846 server_base.cc:1034] running on GCE node
I20260811 22:17:42.313176 12846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:42.314682 12846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:42.315908 12846 hybrid_clock.cc:648] HybridClock initialized: now 1786486662315853 us; error 44 us; skew 500 ppm
I20260811 22:17:42.316290 12846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:42 dist-test-slave-68xk krb5kdc[12826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486662, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:42.319924 12846 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:17:42.321591 12846 webserver.cc:460] Webserver started at http://127.10.173.190:41329/ using document root <none> and password file <none>
I20260811 22:17:42.322296 12846 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:42.322449 12846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:42.322782 12846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:42.325886 12846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "5c3585dccb884d01b6f3b6610529dc61"
format_stamp: "Formatted at 2026-08-11 22:17:42 on dist-test-slave-68xk"
server_key: "41fd587ceead8e7402a01b8edc57c401"
server_key_iv: "9426eeed09efd2096f21c8630ffa6b6e"
server_key_version: "encryptionkey@0"
I20260811 22:17:42.326735 12846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "5c3585dccb884d01b6f3b6610529dc61"
format_stamp: "Formatted at 2026-08-11 22:17:42 on dist-test-slave-68xk"
server_key: "41fd587ceead8e7402a01b8edc57c401"
server_key_iv: "9426eeed09efd2096f21c8630ffa6b6e"
server_key_version: "encryptionkey@0"
I20260811 22:17:42.332046 12846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:17:42.335665 12869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:42.337144 12846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:17:42.337350 12846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "5c3585dccb884d01b6f3b6610529dc61"
format_stamp: "Formatted at 2026-08-11 22:17:42 on dist-test-slave-68xk"
server_key: "41fd587ceead8e7402a01b8edc57c401"
server_key_iv: "9426eeed09efd2096f21c8630ffa6b6e"
server_key_version: "encryptionkey@0"
I20260811 22:17:42.337538 12846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:42.343834 12846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:42.348362 12846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:42.348676 12846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:42.368238 12846 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:41545
I20260811 22:17:42.368287 12955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:41545 every 8 connection(s)
I20260811 22:17:42.369690 12846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:42.370602 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 12846
I20260811 22:17:42.370817 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:42.371358 10934 external_mini_cluster.cc:1428] Setting key 6bd77256c487a45e288a31a4f67dee2b
I20260811 22:17:42.374397 12958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:42.380471 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:42.373267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60491 (local address 127.10.173.190:41545)
0811 22:17:42.374145 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:42.374160 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:42.374214 (+ 54us) server_negotiation.cc:407] Connection header received
0811 22:17:42.376629 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:42.376657 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:42.377054 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:42.377511 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:42.378506 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:42.378514 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:42.378531 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:42.378549 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:42.378637 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:42.378667 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:42.378914 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:42.379052 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 22:17:42.379254 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":337,"threads_started":1}
I20260811 22:17:42.384272 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:42.387632 12958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61: Bootstrap starting.
I20260811 22:17:42.391191 12958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:42.391373 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:42.388842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49447 (local address 127.10.173.190:41545)
0811 22:17:42.389033 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:42.389044 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:42.389340 (+ 296us) server_negotiation.cc:407] Connection header received
0811 22:17:42.389472 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:42.389480 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:42.389541 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:42.389652 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:42.390642 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:42.390649 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:42.390653 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:42.390671 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:42.390715 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:42.390734 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:42.390857 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:42.390932 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 22:17:42.390993 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:17:42.392444 12958 log.cc:824] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:42.395601 12958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61: No bootstrap required, opened a new log
I20260811 22:17:42.403928 12958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3585dccb884d01b6f3b6610529dc61" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41545 } }
I20260811 22:17:42.404317 12958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:42.404456 12958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c3585dccb884d01b6f3b6610529dc61, State: Initialized, Role: FOLLOWER
I20260811 22:17:42.405148 12958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3585dccb884d01b6f3b6610529dc61" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41545 } }
I20260811 22:17:42.405392 12958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:42.405551 12958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:42.405747 12958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:42.408218 12958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3585dccb884d01b6f3b6610529dc61" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41545 } }
I20260811 22:17:42.408841 12958 leader_election.cc:302] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c3585dccb884d01b6f3b6610529dc61; no voters:
I20260811 22:17:42.409523 12958 leader_election.cc:288] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:42.409652 12972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:42.410398 12972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [term 1 LEADER]: Becoming Leader. State: Replica: 5c3585dccb884d01b6f3b6610529dc61, State: Running, Role: LEADER
I20260811 22:17:42.411010 12972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3585dccb884d01b6f3b6610529dc61" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41545 } }
I20260811 22:17:42.411849 12958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:42.414511 12974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c3585dccb884d01b6f3b6610529dc61. Latest consensus state: current_term: 1 leader_uuid: "5c3585dccb884d01b6f3b6610529dc61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3585dccb884d01b6f3b6610529dc61" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41545 } } }
I20260811 22:17:42.414958 12974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:42.415306 12973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c3585dccb884d01b6f3b6610529dc61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c3585dccb884d01b6f3b6610529dc61" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 41545 } } }
I20260811 22:17:42.416404 12973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:42.417860 12982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:42.424072 12982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:42.433637 12982 catalog_manager.cc:1347] Generated new cluster ID: b328f470027c4ea0af73a10b82a47bf9
I20260811 22:17:42.433777 12982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:42.454447 12982 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:42.454979 12982 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:42.462383 12982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c3585dccb884d01b6f3b6610529dc61: Generated new TSK 0
I20260811 22:17:42.463250 12982 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:42.467468 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12846
W20260811 22:17:42.468681 12948 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60491
W20260811 22:17:42.469311 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 12846: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:42.469466 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 12846
2026-08-11T22:17:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:42 dist-test-slave-68xk krb5kdc[12999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:42 dist-test-slave-68xk krb5kdc[12999](info): set up 2 sockets
Aug 11 22:17:42 dist-test-slave-68xk krb5kdc[12999](info): commencing operation
krb5kdc: starting...
W20260811 22:17:43.859040 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.007s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:44 dist-test-slave-68xk krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:17:44.181283 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:36197
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:44437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:36197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:44.524816 13017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:44.525125 13017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:44.525271 13017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:44.532435 13017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:44.532560 13017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:44.532657 13017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:44.532756 13017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:44.532829 13017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:44.542821 13017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:44437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:36197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:36197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:44.544211 13017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:44.545555 13017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:44.557483 13023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:44.557570 13027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:44.557674 13024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:44.558687 13017 server_base.cc:1034] running on GCE node
I20260811 22:17:44.559510 13017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:44.560931 13017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:44.562175 13017 hybrid_clock.cc:648] HybridClock initialized: now 1786486664562104 us; error 65 us; skew 500 ppm
I20260811 22:17:44.562583 13017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:44 dist-test-slave-68xk krb5kdc[12999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486664, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:44.565824 13017 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:17:44.567435 13017 webserver.cc:460] Webserver started at http://127.10.173.190:33679/ using document root <none> and password file <none>
I20260811 22:17:44.568135 13017 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:44.568285 13017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:44.568583 13017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:44.571684 13017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "d07e5fc40ede40dca0b485b0e2df6b2c"
format_stamp: "Formatted at 2026-08-11 22:17:44 on dist-test-slave-68xk"
server_key: "7f24f75008db37f3c92e62a145afb83a"
server_key_iv: "fc76972754c4a762064a88f3e1d52efa"
server_key_version: "encryptionkey@0"
I20260811 22:17:44.572456 13017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "d07e5fc40ede40dca0b485b0e2df6b2c"
format_stamp: "Formatted at 2026-08-11 22:17:44 on dist-test-slave-68xk"
server_key: "7f24f75008db37f3c92e62a145afb83a"
server_key_iv: "fc76972754c4a762064a88f3e1d52efa"
server_key_version: "encryptionkey@0"
I20260811 22:17:44.577672 13017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:17:44.581199 13036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:44.582724 13017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:17:44.582919 13017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "d07e5fc40ede40dca0b485b0e2df6b2c"
format_stamp: "Formatted at 2026-08-11 22:17:44 on dist-test-slave-68xk"
server_key: "7f24f75008db37f3c92e62a145afb83a"
server_key_iv: "fc76972754c4a762064a88f3e1d52efa"
server_key_version: "encryptionkey@0"
I20260811 22:17:44.583122 13017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:44.622336 13017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:44.627084 13017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:44.627420 13017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:44.646996 13017 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:36197
I20260811 22:17:44.647027 13112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:36197 every 8 connection(s)
I20260811 22:17:44.648473 13017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:44.652303 13114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:44.654078 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 13017
I20260811 22:17:44.654279 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:44.654882 10934 external_mini_cluster.cc:1428] Setting key 550edd7a22f11dd9e304488b6f859210
I20260811 22:17:44.666931 13114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c: Bootstrap starting.
Aug 11 22:17:44 dist-test-slave-68xk krb5kdc[12999](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:17:44.670923 13114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:44.671974 13114 log.cc:824] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:44.675772 13114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c: No bootstrap required, opened a new log
I20260811 22:17:44.680155 13120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:44.656747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52373 (local address 127.10.173.190:36197)
0811 22:17:44.657631 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:44.657646 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:44.657689 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:17:44.659408 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:44.659431 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:44.659837 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:44.660162 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:44.661997 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:44.663598 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:44.665534 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:44.666209 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:44.666360 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:44.671447 (+ 5087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:44.671455 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:44.671470 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:44.671512 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:44.675543 (+ 4031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:44.676281 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:44.676290 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:44.676298 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:44.676514 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:44.676881 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:44.676885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:44.676888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:44.677625 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:44.677955 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:44.678276 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 22:17:44.678772 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":421,"threads_started":1}
I20260811 22:17:44.683945 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:44.685628 13114 raft_consensus.cc:348] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07e5fc40ede40dca0b485b0e2df6b2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36197 } }
I20260811 22:17:44.686007 13114 raft_consensus.cc:374] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:44.686105 13114 raft_consensus.cc:729] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07e5fc40ede40dca0b485b0e2df6b2c, State: Initialized, Role: FOLLOWER
I20260811 22:17:44.686832 13114 consensus_queue.cc:260] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07e5fc40ede40dca0b485b0e2df6b2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36197 } }
I20260811 22:17:44.687215 13114 raft_consensus.cc:388] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:44.687371 13114 raft_consensus.cc:482] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:44.687597 13114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:44.691078 13114 raft_consensus.cc:504] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07e5fc40ede40dca0b485b0e2df6b2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36197 } }
I20260811 22:17:44.691743 13114 leader_election.cc:302] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d07e5fc40ede40dca0b485b0e2df6b2c; no voters:
I20260811 22:17:44.692381 13114 leader_election.cc:288] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:44.692516 13131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:44.693769 13131 raft_consensus.cc:686] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [term 1 LEADER]: Becoming Leader. State: Replica: d07e5fc40ede40dca0b485b0e2df6b2c, State: Running, Role: LEADER
I20260811 22:17:44.694449 13131 consensus_queue.cc:237] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07e5fc40ede40dca0b485b0e2df6b2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36197 } }
I20260811 22:17:44.694614 13114 sys_catalog.cc:564] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:44.697649 13132 sys_catalog.cc:455] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d07e5fc40ede40dca0b485b0e2df6b2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07e5fc40ede40dca0b485b0e2df6b2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36197 } } }
I20260811 22:17:44.697633 13133 sys_catalog.cc:455] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d07e5fc40ede40dca0b485b0e2df6b2c. Latest consensus state: current_term: 1 leader_uuid: "d07e5fc40ede40dca0b485b0e2df6b2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d07e5fc40ede40dca0b485b0e2df6b2c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 36197 } } }
I20260811 22:17:44.698184 13132 sys_catalog.cc:458] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:44.698227 13133 sys_catalog.cc:458] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:44.708791 13120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:44.688583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37365 (local address 127.10.173.190:36197)
0811 22:17:44.688806 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:44.688815 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:44.689359 (+ 544us) server_negotiation.cc:407] Connection header received
0811 22:17:44.689509 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:44.689513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:44.689577 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:44.689658 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:44.691669 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:44.692700 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:44.694756 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:44.695210 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:44.695307 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:44.698817 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:44.698828 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:44.698835 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:44.698887 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:44.703771 (+ 4884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:44.704952 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:44.704961 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:44.704967 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:44.705166 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:44.705693 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:44.705701 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:44.705707 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:44.706264 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:44.706599 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:44.708012 (+ 1413us) server_negotiation.cc:299] Negotiation successful
0811 22:17:44.708302 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:17:44.710471 13138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:44.716012 13138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:44.725319 13138 catalog_manager.cc:1347] Generated new cluster ID: e2092eb55e974ff6aa8a9338a2d02fa1
I20260811 22:17:44.725435 13138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:44.761780 13138 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:44.763445 13138 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:44.775916 13138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d07e5fc40ede40dca0b485b0e2df6b2c: Generated new TSK 0
I20260811 22:17:44.776778 13138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:44.793089 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13017
I20260811 22:17:44.794684 13107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:44.892306 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 13017
2026-08-11T22:17:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:45 dist-test-slave-68xk krb5kdc[13163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:45 dist-test-slave-68xk krb5kdc[13163](info): set up 2 sockets
Aug 11 22:17:45 dist-test-slave-68xk krb5kdc[13163](info): commencing operation
krb5kdc: starting...
W20260811 22:17:46.264693 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:46 dist-test-slave-68xk krb5kdc[13163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486666, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:17:46.598543 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:44231
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:37069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:44231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:46.948369 13186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:46.948737 13186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:46.948902 13186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:46.956073 13186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:46.956233 13186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:46.956337 13186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:46.956450 13186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:46.956532 13186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:46.966343 13186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:37069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:44231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:44231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:46.967794 13186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:46.969163 13186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:46.981297 13196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:46.981346 13200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:46.981372 13203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:46.982702 13186 server_base.cc:1034] running on GCE node
I20260811 22:17:46.983440 13186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:46.984777 13186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:46.986014 13186 hybrid_clock.cc:648] HybridClock initialized: now 1786486666985973 us; error 52 us; skew 500 ppm
I20260811 22:17:46.986378 13186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:46 dist-test-slave-68xk krb5kdc[13163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486666, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:46.994729 13186 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:17:46.996354 13186 webserver.cc:460] Webserver started at http://127.10.173.190:37471/ using document root <none> and password file <none>
I20260811 22:17:46.997031 13186 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:46.997135 13186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:46.997400 13186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:47.000562 13186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "566313b5333f49628e4124173c3eddc0"
format_stamp: "Formatted at 2026-08-11 22:17:46 on dist-test-slave-68xk"
server_key: "0b89ccad23301a24d9928518d3c206ec"
server_key_iv: "3060131a877cc57167eb4dcea9f9a961"
server_key_version: "encryptionkey@0"
I20260811 22:17:47.001307 13186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "566313b5333f49628e4124173c3eddc0"
format_stamp: "Formatted at 2026-08-11 22:17:46 on dist-test-slave-68xk"
server_key: "0b89ccad23301a24d9928518d3c206ec"
server_key_iv: "3060131a877cc57167eb4dcea9f9a961"
server_key_version: "encryptionkey@0"
I20260811 22:17:47.007542 13186 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260811 22:17:47.012601 13215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:47.014132 13186 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 22:17:47.014348 13186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "566313b5333f49628e4124173c3eddc0"
format_stamp: "Formatted at 2026-08-11 22:17:46 on dist-test-slave-68xk"
server_key: "0b89ccad23301a24d9928518d3c206ec"
server_key_iv: "3060131a877cc57167eb4dcea9f9a961"
server_key_version: "encryptionkey@0"
I20260811 22:17:47.014618 13186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:47.047904 13186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:47.052513 13186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:47.052809 13186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:47.073159 13186 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:44231
I20260811 22:17:47.073207 13287 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:44231 every 8 connection(s)
I20260811 22:17:47.074718 13186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:47.078627 13290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:47.082480 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 13186
I20260811 22:17:47.082700 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:47.083176 10934 external_mini_cluster.cc:1428] Setting key 21a3e687091a300ef3b8af32f9e82cc6
I20260811 22:17:47.092376 13290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0: Bootstrap starting.
I20260811 22:17:47.096202 13290 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:47.097611 13290 log.cc:824] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:47.098359 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:47.084980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44999 (local address 127.10.173.190:44231)
0811 22:17:47.085848 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:47.085863 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:47.085914 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:17:47.087857 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:47.087892 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:47.088438 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:47.088893 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:47.089442 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:47.091463 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:47.092878 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:47.093867 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:47.094087 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:47.095413 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:47.095424 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:47.095445 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:47.096139 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:47.096255 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:47.096293 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:47.096419 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:47.096701 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 22:17:47.096974 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":352,"threads_started":1}
I20260811 22:17:47.100649 13290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0: No bootstrap required, opened a new log
I20260811 22:17:47.102116 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:47.109102 13290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566313b5333f49628e4124173c3eddc0" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44231 } }
I20260811 22:17:47.109594 13290 raft_consensus.cc:374] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:47.109710 13290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 566313b5333f49628e4124173c3eddc0, State: Initialized, Role: FOLLOWER
I20260811 22:17:47.110363 13290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566313b5333f49628e4124173c3eddc0" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44231 } }
I20260811 22:17:47.110714 13290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:47.110873 13290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:47.111122 13290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:47.113680 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:47.106677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42353 (local address 127.10.173.190:44231)
0811 22:17:47.106918 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:47.106927 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:47.107342 (+ 415us) server_negotiation.cc:407] Connection header received
0811 22:17:47.107567 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:47.107572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:47.107637 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:47.107724 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:47.108376 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:47.109102 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:47.111087 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:47.111620 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:47.111727 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:47.112957 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:47.112961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:47.112964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:47.112985 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:47.113022 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:47.113037 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:47.113120 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:47.113288 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:17:47.113358 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:17:47.114197 13290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566313b5333f49628e4124173c3eddc0" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44231 } }
I20260811 22:17:47.114754 13290 leader_election.cc:302] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 566313b5333f49628e4124173c3eddc0; no voters:
I20260811 22:17:47.115413 13290 leader_election.cc:288] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:47.115535 13306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:47.116300 13306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [term 1 LEADER]: Becoming Leader. State: Replica: 566313b5333f49628e4124173c3eddc0, State: Running, Role: LEADER
I20260811 22:17:47.116928 13306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566313b5333f49628e4124173c3eddc0" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44231 } }
I20260811 22:17:47.117761 13290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:47.121109 13307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "566313b5333f49628e4124173c3eddc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566313b5333f49628e4124173c3eddc0" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44231 } } }
I20260811 22:17:47.121519 13307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:47.121997 13309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 566313b5333f49628e4124173c3eddc0. Latest consensus state: current_term: 1 leader_uuid: "566313b5333f49628e4124173c3eddc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566313b5333f49628e4124173c3eddc0" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 44231 } } }
I20260811 22:17:47.122376 13309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:47.122440 13318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:47.130077 13318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:47.140460 13318 catalog_manager.cc:1347] Generated new cluster ID: 18999923d3424238aaa499844a5145e6
I20260811 22:17:47.140595 13318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:47.155050 13318 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:47.156548 13318 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:47.167212 13318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 566313b5333f49628e4124173c3eddc0: Generated new TSK 0
I20260811 22:17:47.168025 13318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:47.178925 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13186
W20260811 22:17:47.180042 13282 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44999
W20260811 22:17:47.180905 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13186: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:47.181072 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 13186
2026-08-11T22:17:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:47 dist-test-slave-68xk krb5kdc[13339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:47 dist-test-slave-68xk krb5kdc[13339](info): set up 2 sockets
Aug 11 22:17:47 dist-test-slave-68xk krb5kdc[13339](info): commencing operation
krb5kdc: starting...
W20260811 22:17:48.552381 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:48 dist-test-slave-68xk krb5kdc[13339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:17:48.894359 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:42599
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:42599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:49.252565 13363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:49.252844 13363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:49.253029 13363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:49.260202 13363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:49.260310 13363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:49.260426 13363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:49.260535 13363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:49.260622 13363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:49.270232 13363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:42599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:42599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:49.271586 13363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:49.272912 13363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:49.285658 13379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:49.285782 13373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:49.285706 13374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:49.286891 13363 server_base.cc:1034] running on GCE node
I20260811 22:17:49.287802 13363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:49.289445 13363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:49.290731 13363 hybrid_clock.cc:648] HybridClock initialized: now 1786486669290684 us; error 75 us; skew 500 ppm
I20260811 22:17:49.291141 13363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:49 dist-test-slave-68xk krb5kdc[13339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486669, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:49.294466 13363 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:17:49.295974 13363 webserver.cc:460] Webserver started at http://127.10.173.190:36133/ using document root <none> and password file <none>
I20260811 22:17:49.296684 13363 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:49.296837 13363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:49.297163 13363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:49.300293 13363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "81149d5114ad4dd7ac987913d2d1a94e"
format_stamp: "Formatted at 2026-08-11 22:17:49 on dist-test-slave-68xk"
server_key: "a02586cd9bb9d5f29bac897b4f7ae460"
server_key_iv: "23621b50782464da1e97cde052e41971"
server_key_version: "encryptionkey@0"
I20260811 22:17:49.301074 13363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "81149d5114ad4dd7ac987913d2d1a94e"
format_stamp: "Formatted at 2026-08-11 22:17:49 on dist-test-slave-68xk"
server_key: "a02586cd9bb9d5f29bac897b4f7ae460"
server_key_iv: "23621b50782464da1e97cde052e41971"
server_key_version: "encryptionkey@0"
I20260811 22:17:49.306326 13363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:17:49.309847 13386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:49.311357 13363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:17:49.311573 13363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "81149d5114ad4dd7ac987913d2d1a94e"
format_stamp: "Formatted at 2026-08-11 22:17:49 on dist-test-slave-68xk"
server_key: "a02586cd9bb9d5f29bac897b4f7ae460"
server_key_iv: "23621b50782464da1e97cde052e41971"
server_key_version: "encryptionkey@0"
I20260811 22:17:49.311868 13363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:49.317698 13363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:49.321825 13363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:49.322146 13363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:49.341594 13363 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:42599
I20260811 22:17:49.341626 13463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:42599 every 8 connection(s)
I20260811 22:17:49.343093 13363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:49.346920 13464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:49.349069 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 13363
I20260811 22:17:49.349259 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:49.349828 10934 external_mini_cluster.cc:1428] Setting key 8a0face7b193ffd8b186a3516550ce4a
I20260811 22:17:49.357846 13469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:49.351719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51225 (local address 127.10.173.190:42599)
0811 22:17:49.352613 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:49.352624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:49.352668 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:17:49.354337 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:49.354358 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:49.354751 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:49.355065 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:49.356202 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:49.356209 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:49.356221 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:49.356237 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:49.356309 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:49.356334 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:49.356523 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:49.356645 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 22:17:49.356886 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":469,"threads_started":1}
I20260811 22:17:49.359335 13464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e: Bootstrap starting.
I20260811 22:17:49.361639 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:49.362771 13464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:49.364017 13464 log.cc:824] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:49.367300 13464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e: No bootstrap required, opened a new log
I20260811 22:17:49.369712 13469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:49.367161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56009 (local address 127.10.173.190:42599)
0811 22:17:49.367357 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:49.367363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:49.367711 (+ 348us) server_negotiation.cc:407] Connection header received
0811 22:17:49.367873 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:49.367879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:49.367940 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:49.368070 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:49.369002 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:49.369008 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:49.369013 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:17:49.369033 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:49.369085 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:17:49.369109 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:49.369244 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:49.369301 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 22:17:49.369365 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:17:49.376816 13464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81149d5114ad4dd7ac987913d2d1a94e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42599 } }
I20260811 22:17:49.377336 13464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:17:49.377471 13464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81149d5114ad4dd7ac987913d2d1a94e, State: Initialized, Role: FOLLOWER
I20260811 22:17:49.378175 13464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81149d5114ad4dd7ac987913d2d1a94e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42599 } }
I20260811 22:17:49.378465 13464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:17:49.378604 13464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:17:49.378752 13464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:49.381108 13464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81149d5114ad4dd7ac987913d2d1a94e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42599 } }
I20260811 22:17:49.381615 13464 leader_election.cc:302] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81149d5114ad4dd7ac987913d2d1a94e; no voters:
I20260811 22:17:49.382261 13464 leader_election.cc:288] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:17:49.382375 13482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:49.383703 13482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [term 1 LEADER]: Becoming Leader. State: Replica: 81149d5114ad4dd7ac987913d2d1a94e, State: Running, Role: LEADER
I20260811 22:17:49.384248 13482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81149d5114ad4dd7ac987913d2d1a94e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42599 } }
I20260811 22:17:49.384640 13464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:49.387305 13485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81149d5114ad4dd7ac987913d2d1a94e. Latest consensus state: current_term: 1 leader_uuid: "81149d5114ad4dd7ac987913d2d1a94e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81149d5114ad4dd7ac987913d2d1a94e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42599 } } }
I20260811 22:17:49.387760 13485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:49.387706 13483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81149d5114ad4dd7ac987913d2d1a94e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81149d5114ad4dd7ac987913d2d1a94e" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 42599 } } }
I20260811 22:17:49.388006 13483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:49.390863 13494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:49.397006 13494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:49.406946 13494 catalog_manager.cc:1347] Generated new cluster ID: 44c12a64e253425c853df2ef429d8765
I20260811 22:17:49.407084 13494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:49.422446 13494 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:49.422968 13494 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:49.430703 13494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81149d5114ad4dd7ac987913d2d1a94e: Generated new TSK 0
I20260811 22:17:49.431712 13494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:49.442373 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13363
W20260811 22:17:49.443497 13459 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51225
W20260811 22:17:49.444223 10934 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13363: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 22:17:49.444377 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 13363
2026-08-11T22:17:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2267 ms)
[----------] 18 tests from AuthTokenIssuingTest (41869 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:49 dist-test-slave-68xk krb5kdc[13522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:49 dist-test-slave-68xk krb5kdc[13522](info): set up 2 sockets
Aug 11 22:17:49 dist-test-slave-68xk krb5kdc[13522](info): commencing operation
krb5kdc: starting...
W20260811 22:17:50.884152 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.390s user 0.003s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:51 dist-test-slave-68xk krb5kdc[13522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:17:51.238958 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:39303
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:38305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:39303,127.10.173.189:44663,127.10.173.188:37381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:51.591933 13541 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:17:51.592280 13541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:51.592355 13541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:51.592434 13541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:51.599735 13541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:51.599838 13541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:51.599905 13541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:51.599957 13541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:51.600015 13541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:51.609948 13541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:39303,127.10.173.189:44663,127.10.173.188:37381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:39303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:51.611251 13541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:51.612509 13541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:51.624526 13547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:51.624555 13553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:51.624773 13548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:51.625685 13541 server_base.cc:1034] running on GCE node
I20260811 22:17:51.626579 13541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:51.628064 13541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:51.629359 13541 hybrid_clock.cc:648] HybridClock initialized: now 1786486671629293 us; error 53 us; skew 500 ppm
I20260811 22:17:51.629745 13541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:51 dist-test-slave-68xk krb5kdc[13522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486671, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:51.633265 13541 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:17:51.634881 13541 webserver.cc:460] Webserver started at http://127.10.173.190:45815/ using document root <none> and password file <none>
I20260811 22:17:51.635581 13541 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:51.635677 13541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:51.636000 13541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:51.639289 13541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "b2a44d8cbc5340b1b7d03f919878bb4c"
format_stamp: "Formatted at 2026-08-11 22:17:51 on dist-test-slave-68xk"
server_key: "2fcb5e7e7f56e67d904b0418c01a71e1"
server_key_iv: "e49f6a673608d7c0101010c6865aec47"
server_key_version: "encryptionkey@0"
I20260811 22:17:51.640133 13541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "b2a44d8cbc5340b1b7d03f919878bb4c"
format_stamp: "Formatted at 2026-08-11 22:17:51 on dist-test-slave-68xk"
server_key: "2fcb5e7e7f56e67d904b0418c01a71e1"
server_key_iv: "e49f6a673608d7c0101010c6865aec47"
server_key_version: "encryptionkey@0"
I20260811 22:17:51.645701 13541 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 22:17:51.649428 13565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:51.650959 13541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:17:51.651160 13541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "b2a44d8cbc5340b1b7d03f919878bb4c"
format_stamp: "Formatted at 2026-08-11 22:17:51 on dist-test-slave-68xk"
server_key: "2fcb5e7e7f56e67d904b0418c01a71e1"
server_key_iv: "e49f6a673608d7c0101010c6865aec47"
server_key_version: "encryptionkey@0"
I20260811 22:17:51.651398 13541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:51.667464 13541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:51.671556 13541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:51.671840 13541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:51.692204 13541 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:39303
I20260811 22:17:51.692319 13652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:39303 every 8 connection(s)
I20260811 22:17:51.693706 13541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:51.697794 13656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:51.704098 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 13541
I20260811 22:17:51.704368 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:51.705358 10934 external_mini_cluster.cc:1428] Setting key 05e17454557ccc57ba612e32ea305bcb
I20260811 22:17:51.705552 13656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:51 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486671, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:17:51.733847 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:51.709734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:39303 (local address 127.0.0.1:38518)
0811 22:17:51.710758 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:51.710822 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:51.714023 (+ 3201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:51.714991 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:51.715006 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:51.715950 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:51.716525 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:51.716562 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:51.718211 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:51.718219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:51.720340 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:51.720360 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:51.721317 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:51.721324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:51.721459 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:51.722195 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:51.722220 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:51.724738 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:51.729334 (+ 4596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:51.729346 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:51.729356 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:51.729846 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:51.730364 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:51.730368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:51.730370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:51.730652 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:51.731742 (+ 1090us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:51.731749 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:51.731974 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 22:17:51.732222 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 22:17:51.732662 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":396,"threads_started":1}
I20260811 22:17:51.733878 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:51.709734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.10.173.190:39303)
0811 22:17:51.711816 (+ 2082us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:51.711822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:51.711863 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:17:51.714480 (+ 2617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:51.714486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:51.714720 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:51.714885 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:51.716828 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:51.718035 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:51.720494 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:51.721036 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:51.721269 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:51.724994 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:51.725007 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:51.725026 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:51.725085 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:51.729168 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:51.730008 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:51.730020 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:51.730027 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:51.730215 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:51.730783 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:51.730790 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:51.730796 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:51.731593 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:51.731811 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:51.732428 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 22:17:51.732862 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":1884,"thread_start_us":1643,"threads_started":1}
I20260811 22:17:51.737156 13656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:17:51.738518 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:51.738004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:36942)
0811 22:17:51.738205 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:51.738330 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 22:17:51.738814 13573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
I20260811 22:17:51.740442 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:51.740086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:36952)
0811 22:17:51.740235 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:51.740313 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:17:51.740638 13656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:44663: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.10.173.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab.
Entry for principal kudu/127.10.173.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab.
I20260811 22:17:51.782236 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.189:44663
--webserver_interface=127.10.173.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab
--principal=kudu/127.10.173.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:38305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:39303,127.10.173.189:44663,127.10.173.188:37381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:17:51.789978 13656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } attempt: 1
I20260811 22:17:51.791234 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:51.790779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:36968)
0811 22:17:51.790983 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:51.791093 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 22:17:51.792430 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:51.791959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:36978)
0811 22:17:51.792130 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:51.792281 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 22:17:51.792636 13656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:44663: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
I20260811 22:17:51.860917 13656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } attempt: 2
I20260811 22:17:51.862448 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:51.861663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:36984)
0811 22:17:51.862032 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:51.862181 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 22:17:51.863746 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:51.863259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:36998)
0811 22:17:51.863439 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:51.863554 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 22:17:51.863999 13656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:44663: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
I20260811 22:17:51.955312 13656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } attempt: 3
I20260811 22:17:51.956892 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:51.956257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:37004)
0811 22:17:51.956537 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:51.956688 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 22:17:51.958211 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:51.957724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:37006)
0811 22:17:51.957916 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:51.958016 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 22:17:51.958482 13656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:44663: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
I20260811 22:17:52.101791 13656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } attempt: 4
I20260811 22:17:52.103181 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.102508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:37022)
0811 22:17:52.102881 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.103000 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260811 22:17:52.104421 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.103986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:37028)
0811 22:17:52.104162 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.104267 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 22:17:52.104697 13656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:44663: Network error: Client connection negotiation failed: client connection to 127.10.173.189:44663: connect: Connection refused (error 111)
W20260811 22:17:52.151320 13668 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:17:52.151700 13668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:52.151829 13668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:52.151971 13668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:52.159438 13668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:52.159569 13668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:52.159716 13668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:52.159821 13668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:52.159917 13668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:52.169942 13668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:39303,127.10.173.189:44663,127.10.173.188:37381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab
--principal=kudu/127.10.173.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.189:44663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.173.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:52.171345 13668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:52.172711 13668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:52.184696 13681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:52.184670 13678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:52.184731 13679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:52.186136 13668 server_base.cc:1034] running on GCE node
I20260811 22:17:52.187094 13668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:52.188593 13668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:52.189843 13668 hybrid_clock.cc:648] HybridClock initialized: now 1786486672189782 us; error 30 us; skew 500 ppm
I20260811 22:17:52.190275 13668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486672, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:52.198649 13668 init.cc:377] Logged in from keytab as kudu/127.10.173.189@KRBTEST.COM (short username kudu)
I20260811 22:17:52.200476 13668 webserver.cc:460] Webserver started at http://127.10.173.189:42265/ using document root <none> and password file <none>
I20260811 22:17:52.201203 13668 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:52.201305 13668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:52.201591 13668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:52.204713 13668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/data/instance:
uuid: "f31223a77b1047d685f2208722cf2ab2"
format_stamp: "Formatted at 2026-08-11 22:17:52 on dist-test-slave-68xk"
server_key: "f255284d08ce585b4400624fd13447d9"
server_key_iv: "e2b1d36f4228188cdc69a41223b5fe09"
server_key_version: "encryptionkey@0"
I20260811 22:17:52.205492 13668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/wal/instance:
uuid: "f31223a77b1047d685f2208722cf2ab2"
format_stamp: "Formatted at 2026-08-11 22:17:52 on dist-test-slave-68xk"
server_key: "f255284d08ce585b4400624fd13447d9"
server_key_iv: "e2b1d36f4228188cdc69a41223b5fe09"
server_key_version: "encryptionkey@0"
I20260811 22:17:52.210664 13668 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 22:17:52.214037 13690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:52.215474 13668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:17:52.215765 13668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/wal
uuid: "f31223a77b1047d685f2208722cf2ab2"
format_stamp: "Formatted at 2026-08-11 22:17:52 on dist-test-slave-68xk"
server_key: "f255284d08ce585b4400624fd13447d9"
server_key_iv: "e2b1d36f4228188cdc69a41223b5fe09"
server_key_version: "encryptionkey@0"
I20260811 22:17:52.215993 13668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:52.250746 13668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:52.255496 13668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:52.255789 13668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:52.275708 13668 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.189:44663
I20260811 22:17:52.275755 13776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.189:44663 every 8 connection(s)
I20260811 22:17:52.277150 13668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
I20260811 22:17:52.281311 13777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:52.286502 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 13668
I20260811 22:17:52.286733 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-1/wal/instance
I20260811 22:17:52.287254 10934 external_mini_cluster.cc:1428] Setting key d87f026722e472716e2a4865fb1e6df3
I20260811 22:17:52.288867 13777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486672, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:17:52.320788 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.291829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38528 (local address 127.10.173.190:39303)
0811 22:17:52.302763 (+ 10934us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:52.302773 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:52.303336 (+ 563us) server_negotiation.cc:407] Connection header received
0811 22:17:52.305644 (+ 2308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:52.305651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:52.305751 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:52.305890 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:52.307724 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.308574 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.310394 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.310855 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.310993 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.314525 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:52.314530 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:52.314534 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:52.314581 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:52.317389 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.318151 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.318157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.318166 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.318292 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.318892 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.318899 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.318903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.319204 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:52.319386 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:52.320157 (+ 771us) server_negotiation.cc:299] Negotiation successful
0811 22:17:52.320388 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10756,"thread_start_us":10621,"threads_started":1}
I20260811 22:17:52.321777 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.292411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:39303 (local address 127.0.0.1:38528)
0811 22:17:52.303130 (+ 10719us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.303174 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:52.305322 (+ 2148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:52.306161 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:52.306170 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:52.306828 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:52.307455 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.307473 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.308759 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.308764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.310089 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.310103 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.311085 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.311094 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.311371 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.312111 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:52.312134 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:52.314277 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:52.317538 (+ 3261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.317548 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.317563 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.318012 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.318407 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.318434 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.318437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.318717 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.319355 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:52.319363 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:52.319725 (+ 362us) client_negotiation.cc:769] Sending connection context
0811 22:17:52.320050 (+ 325us) client_negotiation.cc:240] Negotiation successful
0811 22:17:52.320584 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10447,"thread_start_us":10215,"threads_started":1}
WARNING: no policy specified for kudu/127.10.173.188@KRBTEST.COM; defaulting to no policy
I20260811 22:17:52.323666 13777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.188@KRBTEST.COM" created.
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486672, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
I20260811 22:17:52.338307 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.324551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:37036)
0811 22:17:52.324789 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.324811 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:52.324937 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:52.326106 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:52.326110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:52.326629 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:52.326760 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.326769 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.328150 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.328157 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.328991 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.329001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.329820 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.329827 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.330021 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.330650 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:52.330667 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:52.332587 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:52.335790 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.335795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.335799 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.336163 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.336532 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.336538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.336541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.336683 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.337274 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:52.337282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:52.337439 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 22:17:52.337586 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:17:52.337877 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 22:17:52.338392 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.324859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.10.173.189:44663)
0811 22:17:52.325703 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:52.325708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:52.325734 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:17:52.325775 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:52.325780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:52.325953 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:52.326106 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:52.326939 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.328019 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.329134 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.329676 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.329812 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.332789 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:52.332799 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:52.332812 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:52.332843 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:52.335634 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.336296 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.336306 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.336314 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.336424 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.336787 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.336794 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.336797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.337151 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:52.337367 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:52.337734 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 22:17:52.337987 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":656,"thread_start_us":498,"threads_started":1}
I20260811 22:17:52.341189 13777 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:17:52.342554 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.341920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46456)
0811 22:17:52.342151 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.342296 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 22:17:52.342823 13694 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
I20260811 22:17:52.344482 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.344112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46472)
0811 22:17:52.344278 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.344368 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 22:17:52.344731 13777 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37381: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab.
Entry for principal kudu/127.10.173.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab.
I20260811 22:17:52.365767 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.188:37381
--webserver_interface=127.10.173.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab
--principal=kudu/127.10.173.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:38305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:39303,127.10.173.189:44663,127.10.173.188:37381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:17:52.394052 13777 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } attempt: 1
I20260811 22:17:52.395303 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.394814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46476)
0811 22:17:52.395025 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.395149 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 22:17:52.396457 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.396106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46492)
0811 22:17:52.396285 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.396357 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 22:17:52.396741 13777 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37381: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
I20260811 22:17:52.403997 13656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } attempt: 5
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486671, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
I20260811 22:17:52.418381 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.404685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:37042)
0811 22:17:52.404868 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.404894 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:52.405000 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:52.405424 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:52.405427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:52.405802 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:52.405984 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.405992 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.407532 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.407536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.408387 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.408399 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.409428 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.409432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.409604 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.410232 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:52.410249 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:52.412393 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:52.416114 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.416118 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.416121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.416481 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.416878 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.416881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.416883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.417010 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.417636 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:52.417640 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:52.417774 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 22:17:52.417908 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 22:17:52.418137 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 22:17:52.418588 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.404752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37042 (local address 127.10.173.189:44663)
0811 22:17:52.404951 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:52.404957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:52.404984 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:17:52.405142 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:52.405148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:52.405254 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:52.405403 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:52.406142 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.407396 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.408560 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.409279 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.409417 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.412590 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:52.412597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:52.412601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:52.412668 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:52.415960 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.416594 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.416600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.416603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.416758 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.417111 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.417116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.417119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.417481 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:52.417646 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:52.418048 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 22:17:52.418273 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:17:52.419870 13656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:17:52.421119 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.420529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46504)
0811 22:17:52.420702 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.420952 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:17:52.422041 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.421717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46512)
0811 22:17:52.421855 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.421941 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:17:52.422282 13656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37381: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
I20260811 22:17:52.464990 13777 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } attempt: 2
I20260811 22:17:52.466266 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.465752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46514)
0811 22:17:52.465975 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.466068 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 22:17:52.467414 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.467062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46518)
0811 22:17:52.467225 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.467297 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:17:52.467628 13777 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37381: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
I20260811 22:17:52.473486 13656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } attempt: 1
I20260811 22:17:52.474628 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.474070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46528)
0811 22:17:52.474309 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.474434 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 22:17:52.475723 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.475303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46530)
0811 22:17:52.475497 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.475588 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 22:17:52.475958 13656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37381: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
I20260811 22:17:52.544227 13656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } attempt: 2
I20260811 22:17:52.545825 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.545107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46540)
0811 22:17:52.545421 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.545566 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 22:17:52.547320 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.546716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46552)
0811 22:17:52.546983 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.547141 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 22:17:52.547681 13656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37381: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
I20260811 22:17:52.558835 13777 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } attempt: 3
I20260811 22:17:52.560138 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.559638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46560)
0811 22:17:52.559878 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.559975 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 22:17:52.561358 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.560888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46568)
0811 22:17:52.561157 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.561244 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260811 22:17:52.561568 13777 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37381: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
I20260811 22:17:52.653934 13656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } attempt: 3
I20260811 22:17:52.655361 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.654867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46574)
0811 22:17:52.655100 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.655190 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 22:17:52.656718 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.656194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46586)
0811 22:17:52.656411 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.656531 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 22:17:52.657001 13656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37381: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
I20260811 22:17:52.704782 13777 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } attempt: 4
I20260811 22:17:52.706105 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.705447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46592)
0811 22:17:52.705756 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.705881 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 22:17:52.707358 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.706899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46606)
0811 22:17:52.707092 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.707199 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:17:52.707666 13777 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37381: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
W20260811 22:17:52.739472 13787 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:17:52.739835 13787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:52.739956 13787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:52.740104 13787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:52.747347 13787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:52.747433 13787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:52.747539 13787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:52.747641 13787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:52.747738 13787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:52.757680 13787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:38305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:39303,127.10.173.189:44663,127.10.173.188:37381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab
--principal=kudu/127.10.173.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.188:37381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.173.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:52.759016 13787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:52.760346 13787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:52.772246 13796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:52.772282 13799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:52.772639 13797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:52.773641 13787 server_base.cc:1034] running on GCE node
I20260811 22:17:52.774544 13787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:52.776012 13787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:52.777288 13787 hybrid_clock.cc:648] HybridClock initialized: now 1786486672777215 us; error 43 us; skew 500 ppm
I20260811 22:17:52.777673 13787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486672, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:52.781239 13787 init.cc:377] Logged in from keytab as kudu/127.10.173.188@KRBTEST.COM (short username kudu)
I20260811 22:17:52.782752 13787 webserver.cc:460] Webserver started at http://127.10.173.188:46617/ using document root <none> and password file <none>
I20260811 22:17:52.783418 13787 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:52.783511 13787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:52.783799 13787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:52.786870 13787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/data/instance:
uuid: "12cbcf280ef3468093756eaf95fee0db"
format_stamp: "Formatted at 2026-08-11 22:17:52 on dist-test-slave-68xk"
server_key: "3b22708b7ba03e29fb05a8eb43eb9954"
server_key_iv: "aa8407a306d324bb415e417e763cd6b8"
server_key_version: "encryptionkey@0"
I20260811 22:17:52.787647 13787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/wal/instance:
uuid: "12cbcf280ef3468093756eaf95fee0db"
format_stamp: "Formatted at 2026-08-11 22:17:52 on dist-test-slave-68xk"
server_key: "3b22708b7ba03e29fb05a8eb43eb9954"
server_key_iv: "aa8407a306d324bb415e417e763cd6b8"
server_key_version: "encryptionkey@0"
I20260811 22:17:52.792932 13787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 22:17:52.796494 13810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:52.797811 13787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:17:52.798139 13787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/wal
uuid: "12cbcf280ef3468093756eaf95fee0db"
format_stamp: "Formatted at 2026-08-11 22:17:52 on dist-test-slave-68xk"
server_key: "3b22708b7ba03e29fb05a8eb43eb9954"
server_key_iv: "aa8407a306d324bb415e417e763cd6b8"
server_key_version: "encryptionkey@0"
I20260811 22:17:52.798373 13787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:52.833801 13787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:52.834298 13656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } attempt: 4
I20260811 22:17:52.835508 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.835063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46616)
0811 22:17:52.835260 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.835357 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:17:52.836503 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.836162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46630)
0811 22:17:52.836308 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.836388 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:17:52.836748 13656 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37381: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37381: connect: Connection refused (error 111)
I20260811 22:17:52.838925 13787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:52.839214 13787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:52.858678 13787 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.188:37381
I20260811 22:17:52.858702 13894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.188:37381 every 8 connection(s)
I20260811 22:17:52.860176 13787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
I20260811 22:17:52.860718 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 13787
I20260811 22:17:52.860913 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486285256683-10934-0/minicluster-data/master-2/wal/instance
I20260811 22:17:52.861599 10934 external_mini_cluster.cc:1428] Setting key 11085aa1518a1403d12f82c169c1b37e
I20260811 22:17:52.864868 13895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:52.871845 13895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486672, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:17:52.885951 13900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.863604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46636 (local address 127.10.173.188:37381)
0811 22:17:52.864380 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:52.864392 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:52.864432 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:17:52.866107 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:52.866130 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:52.866579 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:52.866966 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:52.868502 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.869869 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.871201 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.871856 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.872077 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.876744 (+ 4667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:52.876755 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:52.876767 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:52.876809 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:52.880789 (+ 3980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.881789 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.881802 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.881817 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.882064 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.882558 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.882565 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.882570 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.883238 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:52.883503 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:52.884072 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 22:17:52.884620 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":343,"threads_started":1}
I20260811 22:17:52.893445 13902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.874320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:39303 (local address 127.0.0.1:38542)
0811 22:17:52.875462 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.875506 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:52.875671 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:52.876128 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:52.876135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:52.876948 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:52.877337 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.877350 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.878794 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.878801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.880637 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.880656 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.881858 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.881864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.882065 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.882854 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:52.882872 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:52.885744 (+ 2872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:52.890495 (+ 4751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.890507 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.890520 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.891209 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.891657 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.891663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.891669 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.891851 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.892492 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:52.892498 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:52.892672 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 22:17:52.892832 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 22:17:52.893133 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":884,"thread_start_us":675,"threads_started":1}
I20260811 22:17:52.893970 13901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.874364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38542 (local address 127.10.173.190:39303)
0811 22:17:52.875066 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:52.875073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:52.875627 (+ 554us) server_negotiation.cc:407] Connection header received
0811 22:17:52.875857 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:52.875861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:52.875938 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:52.876060 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:52.877579 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.878561 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.880874 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.881560 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.881751 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.886009 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:52.886016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:52.886020 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:52.886065 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:52.890189 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.891361 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.891366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.891368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.891517 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.891983 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.891987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.891989 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.892313 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:52.892507 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:52.893003 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 22:17:52.893244 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":323,"threads_started":1}
I20260811 22:17:52.895416 13895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486672, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
I20260811 22:17:52.907994 13903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.889716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38546 (local address 127.10.173.190:39303)
0811 22:17:52.890494 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:52.890500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:52.890531 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:17:52.890575 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:52.890579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:52.890643 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:52.890733 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:52.892456 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.893665 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.895324 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.895840 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.895982 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.901237 (+ 5255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:52.901252 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:52.901257 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:52.901308 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:52.904897 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.905637 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.905645 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.905651 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.905849 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.906239 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.906245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.906248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.906669 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:52.906965 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:52.907311 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 22:17:52.907551 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":370,"threads_started":1}
I20260811 22:17:52.911211 13902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.896249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:44663 (local address 127.0.0.1:37052)
0811 22:17:52.896597 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.896635 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:52.896791 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:52.897207 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:52.897212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:52.897726 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:52.897950 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.897965 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.899078 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.899084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.900181 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.900194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.901267 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.901277 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.901538 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.902277 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:52.902307 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:52.904881 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:52.908551 (+ 3670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.908556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.908559 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.908945 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.909345 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.909351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.909354 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.909531 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.910176 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:52.910181 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:52.910335 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 22:17:52.910522 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 22:17:52.910839 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261}
I20260811 22:17:52.911979 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.896240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.10.173.189:44663)
0811 22:17:52.896486 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:52.896491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:52.896725 (+ 234us) server_negotiation.cc:407] Connection header received
0811 22:17:52.896984 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:52.896989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:52.897055 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:52.897181 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:52.898114 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.898920 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.900336 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.901056 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.901178 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.905122 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:52.905128 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:52.905131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:52.905168 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:52.908416 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.909070 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.909075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.909079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.909230 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.909673 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.909677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.909682 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.910037 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:52.910188 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:52.911564 (+ 1376us) server_negotiation.cc:299] Negotiation successful
0811 22:17:52.911728 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:17:52.913100 13895 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
Aug 11 22:17:52 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486672, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
I20260811 22:17:52.927340 13902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.913837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46652)
0811 22:17:52.914006 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:52.914030 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:52.914155 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:52.914545 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:52.914548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:52.914911 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:52.915056 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.915064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.916241 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.916246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.917217 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:52.917230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.918246 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.918251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:52.918533 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.919189 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:52.919207 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:52.921535 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:52.924539 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.924545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.924549 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.925075 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.925464 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:52.925467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:52.925469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:52.925617 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:52.926247 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:52.926252 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:52.926446 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 22:17:52.926628 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 22:17:52.926966 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:17:52.927619 13900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.913919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.10.173.188:37381)
0811 22:17:52.914056 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:52.914060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:52.914076 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:17:52.914290 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:52.914294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:52.914366 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:52.914517 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:52.915251 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.916099 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.917363 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.918050 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.918202 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.921721 (+ 3519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:52.921726 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:52.921729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:52.921757 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:52.924397 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.925185 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.925191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.925194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.925349 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.925738 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.925742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.925744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.926096 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:52.926252 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:52.926803 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 22:17:52.927204 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":50}
I20260811 22:17:52.927946 13905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:52.909971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37068 (local address 127.10.173.189:44663)
0811 22:17:52.910927 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:52.910933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:52.910965 (+ 32us) server_negotiation.cc:407] Connection header received
0811 22:17:52.911013 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:52.911020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:52.911089 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:52.911206 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:52.912791 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.914124 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.915715 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:52.916225 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:52.916364 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:52.921572 (+ 5208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:52.921582 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:52.921590 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:52.921635 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:52.924822 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.925666 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.925673 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.925679 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.925834 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:52.926241 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:52.926245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:52.926248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:52.926638 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:52.926915 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:52.927263 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 22:17:52.927532 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":603,"threads_started":1}
I20260811 22:17:52.933552 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:52.933735 13895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db: Bootstrap starting.
I20260811 22:17:52.937194 13895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:52.938587 13895 log.cc:824] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:52.936543 13613 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38546
I20260811 22:17:52.941880 13895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db: No bootstrap required, opened a new log
I20260811 22:17:52.952785 13895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } }
I20260811 22:17:52.953204 13895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12cbcf280ef3468093756eaf95fee0db, State: Initialized, Role: FOLLOWER
I20260811 22:17:52.953753 13895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } }
I20260811 22:17:52.954793 13910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } } }
I20260811 22:17:52.955276 13910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:52.955971 13895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:17:52.966509 13927 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:17:52.966643 13927 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:17:52.963760 13787 master_runner.cc:186] Error getting master registration for 127.10.173.190:39303: OK, instance_id { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" instance_seqno: 1786486671686443 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:17:52.976634 13787 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:17:53.006909 13777 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } attempt: 5
Aug 11 22:17:53 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486672, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
I20260811 22:17:53.026269 13900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:53.008310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46664 (local address 127.10.173.188:37381)
0811 22:17:53.008656 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.008662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.008685 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:17:53.008772 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.008777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.008878 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.009005 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.010997 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.012180 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.014714 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.015441 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.015573 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:53.019933 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:53.019942 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:53.019948 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:53.019997 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:53.023492 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:53.024148 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:53.024155 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:53.024159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:53.024284 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:53.024713 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:53.024719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:53.024722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:53.025095 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:53.025291 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:53.025658 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 22:17:53.025870 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260811 22:17:53.026261 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:53.007809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46664)
0811 22:17:53.008087 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:53.008116 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:53.008326 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:53.009095 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:53.009101 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:53.009741 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:53.009978 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:53.009991 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.012916 (+ 2925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.012922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:53.014506 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:53.014524 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.015633 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.015641 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:53.015888 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:53.016642 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:53.016674 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:53.019690 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:53.023645 (+ 3955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:53.023654 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:53.023659 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:53.024023 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:53.024404 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:53.024408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:53.024412 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:53.024596 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:53.025223 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:53.025228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:53.025392 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 22:17:53.025547 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 22:17:53.025860 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 22:17:53.031651 13777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2: Bootstrap starting.
I20260811 22:17:53.035897 13777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:53.037238 13777 log.cc:824] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:53.040417 13777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2: No bootstrap required, opened a new log
I20260811 22:17:53.051390 13777 raft_consensus.cc:348] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } }
I20260811 22:17:53.051800 13777 raft_consensus.cc:729] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f31223a77b1047d685f2208722cf2ab2, State: Initialized, Role: FOLLOWER
I20260811 22:17:53.052255 13777 consensus_queue.cc:260] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } }
I20260811 22:17:53.053241 13932 sys_catalog.cc:455] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } } }
I20260811 22:17:53.053643 13932 sys_catalog.cc:458] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:53.054405 13777 sys_catalog.cc:564] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:17:53.069553 13948 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:17:53.069808 13948 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:17:53.066148 13668 master_runner.cc:186] Error getting master registration for 127.10.173.190:39303: OK, instance_id { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" instance_seqno: 1786486671686443 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:17:53.077926 13668 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 22:17:53.091461 13825 tablet.cc:2369] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:17:53.114023 13656 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } attempt: 5
Aug 11 22:17:53 dist-test-slave-68xk krb5kdc[13522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486671, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
I20260811 22:17:53.132216 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:53.114856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37381 (local address 127.0.0.1:46676)
0811 22:17:53.115173 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:53.115195 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:53.115348 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:53.115839 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:53.115846 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:53.116426 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:53.116662 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:53.116677 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.118170 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.118176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:53.119734 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:53.119821 (+ 87us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.120937 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.120946 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:53.121157 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:53.121873 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:53.121900 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:53.124869 (+ 2969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:53.129210 (+ 4341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:53.129216 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:53.129220 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:53.129767 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:53.130198 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:53.130204 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:53.130208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:53.130429 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:53.131084 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:53.131089 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:53.131282 (+ 193us) client_negotiation.cc:769] Sending connection context
0811 22:17:53.131440 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 22:17:53.131759 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260811 22:17:53.132243 13900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:53.114949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46676 (local address 127.10.173.188:37381)
0811 22:17:53.115181 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.115187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.115289 (+ 102us) server_negotiation.cc:407] Connection header received
0811 22:17:53.115573 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.115587 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.115675 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.115777 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.116845 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.118017 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.119966 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.120704 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.120840 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:53.125080 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:53.125087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:53.125093 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:53.125134 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:53.129048 (+ 3914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:53.129885 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:53.129890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:53.129895 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:53.130065 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:53.130543 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:53.130553 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:53.130557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:53.130923 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:53.131095 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:53.131569 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 22:17:53.131839 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 22:17:53.139153 13656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c: Bootstrap starting.
I20260811 22:17:53.143508 13656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:53.144898 13656 log.cc:824] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:53.148535 13656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c: No bootstrap required, opened a new log
I20260811 22:17:53.156601 13656 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } }
I20260811 22:17:53.156994 13656 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a44d8cbc5340b1b7d03f919878bb4c, State: Initialized, Role: FOLLOWER
I20260811 22:17:53.157545 13656 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } }
I20260811 22:17:53.164749 13952 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } } }
I20260811 22:17:53.165169 13952 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:53.167063 13656 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:53.167191 13613 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:17:53.167460 13613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:53.171041 13613 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } }
I20260811 22:17:53.173641 13728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f31223a77b1047d685f2208722cf2ab2"
I20260811 22:17:53.174177 13728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [term 0 FOLLOWER]: Advancing to term 1
W20260811 22:17:53.175034 13579 tablet.cc:2369] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:17:53.177989 13728 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2a44d8cbc5340b1b7d03f919878bb4c in term 1.
I20260811 22:17:53.178761 13573 leader_election.cc:302] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2a44d8cbc5340b1b7d03f919878bb4c, f31223a77b1047d685f2208722cf2ab2; no voters:
I20260811 22:17:53.179283 13952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:53.186761 13613 leader_election.cc:288] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [CANDIDATE]: Term 1 election: Requested vote from peers f31223a77b1047d685f2208722cf2ab2 (127.10.173.189:44663), 12cbcf280ef3468093756eaf95fee0db (127.10.173.188:37381)
I20260811 22:17:53.187011 13855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "12cbcf280ef3468093756eaf95fee0db"
I20260811 22:17:53.187556 13855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:53.190173 13952 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [term 1 LEADER]: Becoming Leader. State: Replica: b2a44d8cbc5340b1b7d03f919878bb4c, State: Running, Role: LEADER
I20260811 22:17:53.190786 13952 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } }
I20260811 22:17:53.191212 13855 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2a44d8cbc5340b1b7d03f919878bb4c in term 1.
I20260811 22:17:53.198184 13956 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a44d8cbc5340b1b7d03f919878bb4c. Latest consensus state: current_term: 1 leader_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } } }
I20260811 22:17:53.198719 13956 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:53.202907 13963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:17:53.222237 13963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:53.253382 13855 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [term 1 FOLLOWER]: Refusing update from remote peer b2a44d8cbc5340b1b7d03f919878bb4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:17:53.254765 13728 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [term 1 FOLLOWER]: Refusing update from remote peer b2a44d8cbc5340b1b7d03f919878bb4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:17:53.256157 13956 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:17:53.257835 13956 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:17:53.258807 13697 tablet.cc:2369] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:17:53.271188 13900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:53.228452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46686 (local address 127.10.173.188:37381)
0811 22:17:53.228651 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.228657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.243175 (+ 14518us) server_negotiation.cc:407] Connection header received
0811 22:17:53.243774 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.243783 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.243891 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.243971 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.246235 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.247303 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.249520 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.250206 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.250309 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:53.256127 (+ 5818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:53.256133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:53.256140 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:53.256191 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:53.259912 (+ 3721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:53.268106 (+ 8194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:53.268112 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:53.268115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:53.268298 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:53.268910 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:53.268918 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:53.268924 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:53.269392 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:53.269692 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:53.270374 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 22:17:53.270710 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:17:53.292201 13910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a44d8cbc5340b1b7d03f919878bb4c. Latest consensus state: current_term: 1 leader_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } } }
I20260811 22:17:53.292547 13910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:53.296541 13981 mvcc.cc:205] Tried to move back new op lower bound from 7317449413575647232 to 7317449413395578880. Current Snapshot: MvccSnapshot[applied={T|T < 7317449413575647232}]
I20260811 22:17:53.321197 13903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:53.218694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38562 (local address 127.10.173.190:39303)
0811 22:17:53.219004 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.219011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.227880 (+ 8869us) server_negotiation.cc:407] Connection header received
0811 22:17:53.230561 (+ 2681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.230570 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.230687 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.230782 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.238892 (+ 8110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.239895 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.245299 (+ 5404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.246050 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.246195 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:53.282823 (+ 36628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:53.282831 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:53.282837 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:53.282890 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:53.286980 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:53.318157 (+ 31177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:53.318165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:53.318170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:53.318449 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:53.319060 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:53.319065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:53.319069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:53.319498 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:53.319764 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:53.320480 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 22:17:53.320761 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260811 22:17:53.321183 13905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:53.218761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37084 (local address 127.10.173.189:44663)
0811 22:17:53.219174 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.219180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.238622 (+ 19442us) server_negotiation.cc:407] Connection header received
0811 22:17:53.242018 (+ 3396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.242025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.242141 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.242214 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.250169 (+ 7955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.251369 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.277470 (+ 26101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:53.286654 (+ 9184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:53.286829 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:53.302656 (+ 15827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:53.302668 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:53.302677 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:53.302749 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:53.307213 (+ 4464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:53.308284 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:53.308291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:53.308295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:53.308459 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:53.318565 (+ 10106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:53.318571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:53.318575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:53.319111 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:53.319408 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:53.320259 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 22:17:53.320636 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276}
I20260811 22:17:53.333278 13963 catalog_manager.cc:1347] Generated new cluster ID: f1c9271cce3041368282c8bb428cd297
I20260811 22:17:53.333412 13963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:53.336126 13910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } } }
I20260811 22:17:53.336444 13910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:53.337718 13952 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } } }
I20260811 22:17:53.338029 13952 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:53.342514 13932 sys_catalog.cc:455] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a44d8cbc5340b1b7d03f919878bb4c. Latest consensus state: current_term: 1 leader_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } } }
I20260811 22:17:53.342846 13932 sys_catalog.cc:458] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:53.358918 13956 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } } }
I20260811 22:17:53.359334 13956 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:53.364107 13988 mvcc.cc:205] Tried to move back new op lower bound from 7317449413575647232 to 7317449413395578880. Current Snapshot: MvccSnapshot[applied={T|T < 7317449413575647232}]
I20260811 22:17:53.377996 13932 sys_catalog.cc:455] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2a44d8cbc5340b1b7d03f919878bb4c" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 39303 } } peers { permanent_uuid: "f31223a77b1047d685f2208722cf2ab2" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 44663 } } peers { permanent_uuid: "12cbcf280ef3468093756eaf95fee0db" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37381 } } }
I20260811 22:17:53.378300 13932 sys_catalog.cc:458] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:53.399514 13963 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:53.401597 13963 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:53.437089 14005 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:17:53.450544 13963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2a44d8cbc5340b1b7d03f919878bb4c: Generated new TSK 0
I20260811 22:17:53.451676 13963 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:17:53.607776 13903 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.584344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38572 (local address 127.10.173.190:39303)
0811 22:17:53.584534 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.584541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.605394 (+ 20853us) server_negotiation.cc:407] Connection header received
0811 22:17:53.605644 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.605650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.605777 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.605892 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.607382 (+ 1490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38572: BlockingRecv error: recv got EOF from 127.0.0.1:38572 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 22:17:53.610358 13900 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.606172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46692 (local address 127.10.173.188:37381)
0811 22:17:53.606373 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.606380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.608607 (+ 2227us) server_negotiation.cc:407] Connection header received
0811 22:17:53.608875 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.608881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.608965 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.609045 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.610103 (+ 1058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46692: BlockingRecv error: recv got EOF from 127.0.0.1:46692 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 22:17:53.610893 13905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.590622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.10.173.189:44663)
0811 22:17:53.590832 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.590839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.608590 (+ 17751us) server_negotiation.cc:407] Connection header received
0811 22:17:53.609239 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.609245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.609337 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.609422 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.610626 (+ 1204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37096: BlockingRecv error: recv got EOF from 127.0.0.1:37096 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 22:17:53.620239 13905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.618187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37098 (local address 127.10.173.189:44663)
0811 22:17:53.618397 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.618405 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.618703 (+ 298us) server_negotiation.cc:407] Connection header received
0811 22:17:53.618964 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.618968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.619153 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.619260 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.620000 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37098: BlockingRecv error: recv got EOF from 127.0.0.1:37098 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 22:17:53.628320 13905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.625214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.10.173.189:44663)
0811 22:17:53.625370 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.625374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.626946 (+ 1572us) server_negotiation.cc:407] Connection header received
0811 22:17:53.627182 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.627194 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.627284 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.627389 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.628132 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37114: BlockingRecv error: recv got EOF from 127.0.0.1:37114 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 22:17:53.636368 13905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.633904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37126 (local address 127.10.173.189:44663)
0811 22:17:53.634058 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.634062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.634981 (+ 919us) server_negotiation.cc:407] Connection header received
0811 22:17:53.635202 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.635206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.635272 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.635393 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.636109 (+ 716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37126: BlockingRecv error: recv got EOF from 127.0.0.1:37126 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 22:17:53.647336 13905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.644867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.10.173.189:44663)
0811 22:17:53.645166 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.645175 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.645727 (+ 552us) server_negotiation.cc:407] Connection header received
0811 22:17:53.645975 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.645981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.646066 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.646190 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.647107 (+ 917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37130: BlockingRecv error: recv got EOF from 127.0.0.1:37130 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260811 22:17:53.662009 13905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.659719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37146 (local address 127.10.173.189:44663)
0811 22:17:53.659941 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.659951 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.660522 (+ 571us) server_negotiation.cc:407] Connection header received
0811 22:17:53.660766 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.660772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.660875 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.660993 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.661777 (+ 784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37146: BlockingRecv error: recv got EOF from 127.0.0.1:37146 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 22:17:53.684206 13905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.681611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37158 (local address 127.10.173.189:44663)
0811 22:17:53.681909 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.681918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.682577 (+ 659us) server_negotiation.cc:407] Connection header received
0811 22:17:53.682848 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.682854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.682931 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.683048 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.683876 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37158: BlockingRecv error: recv got EOF from 127.0.0.1:37158 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260811 22:17:53.722113 13905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.719887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.10.173.189:44663)
0811 22:17:53.720083 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.720091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.720547 (+ 456us) server_negotiation.cc:407] Connection header received
0811 22:17:53.720787 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.720794 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.720902 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.721058 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.721861 (+ 803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37166: BlockingRecv error: recv got EOF from 127.0.0.1:37166 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 22:17:53.791812 13905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.789768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37180 (local address 127.10.173.189:44663)
0811 22:17:53.789987 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.789995 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.790301 (+ 306us) server_negotiation.cc:407] Connection header received
0811 22:17:53.790499 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.790506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.790634 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.790768 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.791534 (+ 766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37180: BlockingRecv error: recv got EOF from 127.0.0.1:37180 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 22:17:53.925618 13905 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:53.923773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37188 (local address 127.10.173.189:44663)
0811 22:17:53.923960 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:53.923965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:53.924305 (+ 340us) server_negotiation.cc:407] Connection header received
0811 22:17:53.924472 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:53.924477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:53.924556 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:53.924660 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:53.925371 (+ 711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37188: BlockingRecv error: recv got EOF from 127.0.0.1:37188 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:17:53.968993 13927 catalog_manager.cc:1259] Loaded cluster ID: f1c9271cce3041368282c8bb428cd297
I20260811 22:17:53.969092 13927 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db: loading cluster ID for follower catalog manager: success
I20260811 22:17:53.971627 13927 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db: acquiring CA information for follower catalog manager: success
I20260811 22:17:53.973349 13927 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 12cbcf280ef3468093756eaf95fee0db: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:17:54.072139 13948 catalog_manager.cc:1259] Loaded cluster ID: f1c9271cce3041368282c8bb428cd297
I20260811 22:17:54.072252 13948 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2: loading cluster ID for follower catalog manager: success
I20260811 22:17:54.074922 13948 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2: acquiring CA information for follower catalog manager: success
I20260811 22:17:54.076778 13948 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f31223a77b1047d685f2208722cf2ab2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:17:54.194564 13905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:54.188642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.10.173.189:44663)
0811 22:17:54.188886 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:54.188894 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:54.189127 (+ 233us) server_negotiation.cc:407] Connection header received
0811 22:17:54.189415 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:54.189426 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:54.189582 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:54.189733 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:54.190177 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:54.190934 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:54.192343 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:54.192749 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:54.192889 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:54.193387 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:54.193908 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:54.194076 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:54.194136 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 22:17:54.194220 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:17:54.206727 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:54.200578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.10.173.188:37381)
0811 22:17:54.201112 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:54.201117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:54.201171 (+ 54us) server_negotiation.cc:407] Connection header received
0811 22:17:54.201373 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:54.201377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:54.201578 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:54.201726 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:54.202147 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:54.203019 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:54.204440 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:54.204840 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:54.204935 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:54.205417 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:54.205930 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:54.206039 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:54.206200 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 22:17:54.206323 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":260,"threads_started":1}
I20260811 22:17:54.225888 14116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:54.217790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.10.173.190:39303)
0811 22:17:54.218300 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:54.218308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:54.219728 (+ 1420us) server_negotiation.cc:407] Connection header received
0811 22:17:54.219855 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:54.219861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:54.220029 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:54.220212 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:54.220816 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:54.221557 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:54.223440 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:54.223847 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:54.223995 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:54.224611 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:54.225114 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:54.225242 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:54.225407 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 22:17:54.225512 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":265,"threads_started":1}
I20260811 22:17:54.227438 13905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:54.217550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37200 (local address 127.10.173.189:44663)
0811 22:17:54.217724 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:54.217730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:54.218691 (+ 961us) server_negotiation.cc:407] Connection header received
0811 22:17:54.218903 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:54.218912 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:54.219017 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:54.219123 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:54.220358 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:54.221953 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:54.224858 (+ 2905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:54.225638 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:54.225739 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:54.226583 (+ 844us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:54.226730 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:54.226804 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:54.227093 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 22:17:54.227187 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:17:54.230599 14108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:54.219779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.10.173.188:37381)
0811 22:17:54.220008 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:54.220018 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:54.223044 (+ 3026us) server_negotiation.cc:407] Connection header received
0811 22:17:54.223218 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:54.223227 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:54.223321 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:54.223427 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:54.224285 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:54.225261 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:54.228349 (+ 3088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:54.228952 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:54.229079 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:54.229838 (+ 759us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:54.229940 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:54.230066 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:54.230188 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 22:17:54.230271 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:17:54.230717 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13541
I20260811 22:17:54.231762 13644 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:54.348073 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 13541
I20260811 22:17:54.380194 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13668
I20260811 22:17:54.381458 13766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:54.491475 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 13668
I20260811 22:17:54.522487 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 13787
I20260811 22:17:54.523702 13887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:54.632335 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 13787
2026-08-11T22:17:54Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5189 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:17:54 dist-test-slave-68xk krb5kdc[14133](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:17:54 dist-test-slave-68xk krb5kdc[14133](info): set up 2 sockets
Aug 11 22:17:54 dist-test-slave-68xk krb5kdc[14133](info): commencing operation
krb5kdc: starting...
W20260811 22:17:55.994532 10934 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.007s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/test-user.keytab.
Aug 11 22:17:56 dist-test-slave-68xk krb5kdc[14133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:17:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:17:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.173.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
Entry for principal kudu/127.10.173.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab.
I20260811 22:17:56.342553 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:33897
--webserver_interface=127.10.173.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:35993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:33897,127.10.173.189:42919,127.10.173.188:37937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:17:56.692049 14155 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:17:56.692431 14155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:56.692559 14155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:56.692692 14155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:56.700094 14155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:56.700191 14155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:56.700284 14155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:56.700366 14155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:56.700475 14155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:56.710522 14155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:33897,127.10.173.189:42919,127.10.173.188:37937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.190.keytab
--principal=kudu/127.10.173.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.190:33897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.173.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:56.711840 14155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:56.713193 14155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:56.725443 14164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:56.725576 14163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:56.725598 14166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:56.726212 14155 server_base.cc:1034] running on GCE node
I20260811 22:17:56.726950 14155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:56.728308 14155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:56.729553 14155 hybrid_clock.cc:648] HybridClock initialized: now 1786486676729485 us; error 42 us; skew 500 ppm
I20260811 22:17:56.729890 14155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:56 dist-test-slave-68xk krb5kdc[14133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486676, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:56.733048 14155 init.cc:377] Logged in from keytab as kudu/127.10.173.190@KRBTEST.COM (short username kudu)
I20260811 22:17:56.734658 14155 webserver.cc:460] Webserver started at http://127.10.173.190:42349/ using document root <none> and password file <none>
I20260811 22:17:56.735366 14155 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:56.735502 14155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:56.735827 14155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:56.738862 14155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/data/instance:
uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133"
format_stamp: "Formatted at 2026-08-11 22:17:56 on dist-test-slave-68xk"
server_key: "e0c78f06b2ea1c4057e8759168af9f3c"
server_key_iv: "77892678b8f0dfa3493824e012031f7b"
server_key_version: "encryptionkey@0"
I20260811 22:17:56.739670 14155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/wal/instance:
uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133"
format_stamp: "Formatted at 2026-08-11 22:17:56 on dist-test-slave-68xk"
server_key: "e0c78f06b2ea1c4057e8759168af9f3c"
server_key_iv: "77892678b8f0dfa3493824e012031f7b"
server_key_version: "encryptionkey@0"
I20260811 22:17:56.744946 14155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 22:17:56.748638 14177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:56.750120 14155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:17:56.750331 14155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/wal
uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133"
format_stamp: "Formatted at 2026-08-11 22:17:56 on dist-test-slave-68xk"
server_key: "e0c78f06b2ea1c4057e8759168af9f3c"
server_key_iv: "77892678b8f0dfa3493824e012031f7b"
server_key_version: "encryptionkey@0"
I20260811 22:17:56.750626 14155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:56.768270 14155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:56.772274 14155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:56.772570 14155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:56.791857 14155 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.190:33897
I20260811 22:17:56.791893 14257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.190:33897 every 8 connection(s)
I20260811 22:17:56.793340 14155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/data/info.pb
I20260811 22:17:56.796602 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 14155
I20260811 22:17:56.796801 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-0/wal/instance
I20260811 22:17:56.797396 10934 external_mini_cluster.cc:1428] Setting key caeda52c98c0366a7dc25fbb4285b516
I20260811 22:17:56.797492 14258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:56.807576 14258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:56 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486676, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.173.189@KRBTEST.COM; defaulting to no policy
I20260811 22:17:56.831113 14263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:56.812765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.10.173.190:33897)
0811 22:17:56.814470 (+ 1705us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:56.814478 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:56.814533 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:17:56.815330 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:56.815335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:56.815568 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:56.815772 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:56.817088 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:56.818299 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:56.819691 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:56.820243 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:56.820448 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:56.823886 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:56.823895 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:56.823908 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:56.823937 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:56.826902 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:56.827610 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:56.827617 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:56.827619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:56.827764 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:56.828164 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:56.828176 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:56.828178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:56.828778 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:56.829001 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:56.829706 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 22:17:56.830025 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":1457,"thread_start_us":1296,"threads_started":1}
I20260811 22:17:56.831194 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:56.811490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:33897 (local address 127.0.0.1:46764)
0811 22:17:56.812360 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:56.812407 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:56.814956 (+ 2549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:56.815804 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:56.815811 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:56.816514 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:56.816860 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:56.816888 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:56.818487 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:56.818491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:56.819539 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:56.819551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:56.820488 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:56.820494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:56.820703 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:56.821406 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:56.821430 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:56.823658 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:56.827038 (+ 3380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:56.827050 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:56.827069 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:56.827462 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:56.827879 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:56.827884 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:56.827888 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:56.828043 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:56.828918 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:56.828931 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:56.829181 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 22:17:56.829492 (+ 311us) client_negotiation.cc:240] Negotiation successful
0811 22:17:56.829897 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":317,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.189@KRBTEST.COM" created.
I20260811 22:17:56.834682 14258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:17:56.835872 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:56.835478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35090)
0811 22:17:56.835644 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:56.835765 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 22:17:56.836169 14179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
I20260811 22:17:56.837762 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:56.837398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35092)
0811 22:17:56.837558 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:56.837634 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 22:17:56.837968 14258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:42919: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab.
Entry for principal kudu/127.10.173.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab.
I20260811 22:17:56.874209 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.189:42919
--webserver_interface=127.10.173.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab
--principal=kudu/127.10.173.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:35993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:33897,127.10.173.189:42919,127.10.173.188:37937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:17:56.887290 14258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } attempt: 1
I20260811 22:17:56.888775 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:56.888132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35102)
0811 22:17:56.888412 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:56.888562 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 22:17:56.890054 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:56.889567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35116)
0811 22:17:56.889768 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:56.889872 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 22:17:56.890297 14258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:42919: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
I20260811 22:17:56.958581 14258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } attempt: 2
I20260811 22:17:56.959957 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:56.959353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35120)
0811 22:17:56.959644 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:56.959762 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 22:17:56.961184 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:56.960690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35122)
0811 22:17:56.960899 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:56.961024 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:17:56.961432 14258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:42919: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
I20260811 22:17:57.052744 14258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } attempt: 3
I20260811 22:17:57.054454 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.053690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35136)
0811 22:17:57.054024 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.054165 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 22:17:57.055747 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.055273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35150)
0811 22:17:57.055459 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.055585 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 22:17:57.056053 14258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:42919: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
I20260811 22:17:57.199318 14258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } attempt: 4
I20260811 22:17:57.200664 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.200086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35154)
0811 22:17:57.200432 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.200525 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198}
I20260811 22:17:57.201725 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.201357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35166)
0811 22:17:57.201510 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.201596 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:17:57.201906 14258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.189:42919: Network error: Client connection negotiation failed: client connection to 127.10.173.189:42919: connect: Connection refused (error 111)
W20260811 22:17:57.228227 14267 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:17:57.228607 14267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:57.228741 14267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:57.228885 14267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:57.236317 14267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:57.236441 14267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:57.236553 14267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:57.236644 14267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:57.236734 14267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:57.246436 14267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:33897,127.10.173.189:42919,127.10.173.188:37937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.189.keytab
--principal=kudu/127.10.173.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.189:42919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.173.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:57.247793 14267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:57.249168 14267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:57.261358 14273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:57.261442 14274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:57.261411 14277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:57.262063 14267 server_base.cc:1034] running on GCE node
I20260811 22:17:57.262909 14267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:57.264395 14267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:57.265640 14267 hybrid_clock.cc:648] HybridClock initialized: now 1786486677265568 us; error 50 us; skew 500 ppm
I20260811 22:17:57.265995 14267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:57.273491 14267 init.cc:377] Logged in from keytab as kudu/127.10.173.189@KRBTEST.COM (short username kudu)
I20260811 22:17:57.275163 14267 webserver.cc:460] Webserver started at http://127.10.173.189:42721/ using document root <none> and password file <none>
I20260811 22:17:57.275851 14267 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:57.275940 14267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:57.276211 14267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:57.279400 14267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/data/instance:
uuid: "17060a024cc54a5cbb1aecb6af6aa908"
format_stamp: "Formatted at 2026-08-11 22:17:57 on dist-test-slave-68xk"
server_key: "db1659189392c0bbb8ef532cdec46673"
server_key_iv: "1f358b259e1bbe761395946aca43d87f"
server_key_version: "encryptionkey@0"
I20260811 22:17:57.280455 14267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/wal/instance:
uuid: "17060a024cc54a5cbb1aecb6af6aa908"
format_stamp: "Formatted at 2026-08-11 22:17:57 on dist-test-slave-68xk"
server_key: "db1659189392c0bbb8ef532cdec46673"
server_key_iv: "1f358b259e1bbe761395946aca43d87f"
server_key_version: "encryptionkey@0"
I20260811 22:17:57.287626 14267 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 22:17:57.292109 14288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:57.293525 14267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 22:17:57.293735 14267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/wal
uuid: "17060a024cc54a5cbb1aecb6af6aa908"
format_stamp: "Formatted at 2026-08-11 22:17:57 on dist-test-slave-68xk"
server_key: "db1659189392c0bbb8ef532cdec46673"
server_key_iv: "1f358b259e1bbe761395946aca43d87f"
server_key_version: "encryptionkey@0"
I20260811 22:17:57.293967 14267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:57.308501 14267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:57.312834 14267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:57.313084 14267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:57.334203 14267 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.189:42919
I20260811 22:17:57.334227 14375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.189:42919 every 8 connection(s)
I20260811 22:17:57.335824 14267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/data/info.pb
I20260811 22:17:57.338814 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 14267
I20260811 22:17:57.339037 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-1/wal/instance
I20260811 22:17:57.339684 10934 external_mini_cluster.cc:1428] Setting key f13c7332b9b8ea9192c57906f4ee4c59
I20260811 22:17:57.340471 14376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:57.351568 14376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.173.188@KRBTEST.COM; defaulting to no policy
I20260811 22:17:57.378635 14379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.355375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46780 (local address 127.10.173.190:33897)
0811 22:17:57.355997 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:57.356007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:57.356962 (+ 955us) server_negotiation.cc:407] Connection header received
0811 22:17:57.359995 (+ 3033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:57.359999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:57.360084 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:57.360198 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:57.362499 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.363360 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.365906 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.366355 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.366582 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.371192 (+ 4610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:57.371201 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:57.371209 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:57.371271 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:57.375260 (+ 3989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.376022 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.376030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.376035 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.376190 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.376609 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.376616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.376621 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.377043 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:57.377203 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:57.377899 (+ 696us) server_negotiation.cc:299] Negotiation successful
0811 22:17:57.378128 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":325,"threads_started":1}
I20260811 22:17:57.379449 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.355949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:33897 (local address 127.0.0.1:46780)
0811 22:17:57.356780 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.356826 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:57.359669 (+ 2843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:57.360493 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:57.360507 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:57.361341 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:57.362210 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.362248 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.363547 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.363555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.365732 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.365751 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.366723 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.366735 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.367079 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.367907 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:57.367949 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:57.370924 (+ 2975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:57.375415 (+ 4491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.375423 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.375436 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.375881 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.376319 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.376322 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.376325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.376488 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.377185 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:57.377191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:57.377519 (+ 328us) client_negotiation.cc:769] Sending connection context
0811 22:17:57.377753 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 22:17:57.378127 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":371,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.173.188@KRBTEST.COM" created.
I20260811 22:17:57.381490 14376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
I20260811 22:17:57.401786 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.382430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35172)
0811 22:17:57.382675 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.382696 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:57.382824 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:57.387555 (+ 4731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:57.387562 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:57.387987 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:57.388161 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.388170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.390249 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.390254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.391340 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.391354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.392624 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.392629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.392797 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.393435 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:57.393453 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:57.395704 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:57.399289 (+ 3585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.399297 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.399302 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.399661 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.400063 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.400067 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.400071 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.400215 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.400802 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:57.400806 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:57.400953 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 22:17:57.401085 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 22:17:57.401367 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 22:17:57.401873 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.382673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35172 (local address 127.10.173.189:42919)
0811 22:17:57.386987 (+ 4314us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:57.386997 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:57.387033 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:17:57.387094 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:57.387104 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:57.387367 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:57.387540 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:57.388347 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.390084 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.391495 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.392451 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.392595 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.395955 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:57.395967 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:57.395980 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:57.396008 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:57.399146 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.399818 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.399825 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.399828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.399939 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.400329 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.400339 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.400341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.400675 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:57.400892 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:57.401266 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 22:17:57.401493 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4105,"thread_start_us":3923,"threads_started":1}
I20260811 22:17:57.404522 14376 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:17:57.405715 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.405253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33514)
0811 22:17:57.405446 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.405550 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 22:17:57.406080 14292 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
I20260811 22:17:57.407665 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.407345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33530)
0811 22:17:57.407483 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.407562 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:17:57.408006 14376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37937: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.173.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab.
Entry for principal kudu/127.10.173.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab.
I20260811 22:17:57.422818 10934 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskZXazun/build/asan/bin/kudu
/tmp/dist-test-taskZXazun/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.188:37937
--webserver_interface=127.10.173.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab
--principal=kudu/127.10.173.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.173.148:35993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.173.190:33897,127.10.173.189:42919,127.10.173.188:37937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:17:57.457329 14376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } attempt: 1
I20260811 22:17:57.458688 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.458093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33540)
0811 22:17:57.458324 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.458518 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 22:17:57.459880 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.459488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33544)
0811 22:17:57.459688 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.459769 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:17:57.460139 14376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37937: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
I20260811 22:17:57.501174 14258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } attempt: 5
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486676, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
I20260811 22:17:57.516003 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.502052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35184 (local address 127.10.173.189:42919)
0811 22:17:57.502260 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:57.502265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:57.502286 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:17:57.502515 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:57.502519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:57.502604 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:57.502689 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:57.503696 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.504392 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.506125 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.506639 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.506746 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.510542 (+ 3796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:57.510547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:57.510551 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:57.510585 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:57.513330 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.514000 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.514006 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.514010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.514136 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.514581 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.514587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.514591 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.514878 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:57.515039 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:57.515453 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 22:17:57.515676 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:17:57.516003 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.501946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35184)
0811 22:17:57.502181 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.502204 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:57.502353 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:57.502762 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:57.502767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:57.503285 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:57.503532 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.503549 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.504532 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.504538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.505957 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.505973 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.506807 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.506815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.507094 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.507799 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:57.507822 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:57.510301 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:57.513477 (+ 3176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.513483 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.513487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.513877 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.514262 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.514267 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.514271 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.514464 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.515003 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:57.515008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:57.515170 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 22:17:57.515350 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 22:17:57.515640 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 22:17:57.517130 14258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:17:57.518368 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.517973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33556)
0811 22:17:57.518149 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.518237 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:17:57.519397 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.519052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33560)
0811 22:17:57.519206 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.519290 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:17:57.519613 14258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37937: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
I20260811 22:17:57.528389 14376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } attempt: 2
I20260811 22:17:57.529328 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.528930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33570)
0811 22:17:57.529101 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.529184 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 22:17:57.530316 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.529933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33584)
0811 22:17:57.530110 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.530185 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 22:17:57.530630 14376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37937: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
I20260811 22:17:57.570871 14258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } attempt: 1
I20260811 22:17:57.572131 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.571573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33598)
0811 22:17:57.571790 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.571937 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 22:17:57.573167 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.572840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33606)
0811 22:17:57.572991 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.573068 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:17:57.573402 14258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37937: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
I20260811 22:17:57.621901 14376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } attempt: 3
I20260811 22:17:57.623258 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.622709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33612)
0811 22:17:57.622933 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.623066 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 22:17:57.624598 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.624138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33616)
0811 22:17:57.624388 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.624474 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260811 22:17:57.624933 14376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37937: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
I20260811 22:17:57.641654 14258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } attempt: 2
I20260811 22:17:57.642881 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.642360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33628)
0811 22:17:57.642612 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.642715 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 22:17:57.643884 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.643541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33632)
0811 22:17:57.643694 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.643778 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:17:57.644095 14258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37937: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
I20260811 22:17:57.750339 14258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } attempt: 3
I20260811 22:17:57.751598 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.751072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33646)
0811 22:17:57.751315 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.751419 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 22:17:57.752703 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.752295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33652)
0811 22:17:57.752526 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.752599 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260811 22:17:57.752904 14258 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37937: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
I20260811 22:17:57.768179 14376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } attempt: 4
I20260811 22:17:57.769366 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.768878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33666)
0811 22:17:57.769100 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.769184 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 22:17:57.770339 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.769997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33668)
0811 22:17:57.770158 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.770235 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:17:57.770583 14376 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.173.188:37937: Network error: Client connection negotiation failed: client connection to 127.10.173.188:37937: connect: Connection refused (error 111)
W20260811 22:17:57.779825 14384 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:17:57.780113 14384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:17:57.780176 14384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:17:57.780258 14384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:17:57.787480 14384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:17:57.787567 14384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:17:57.787638 14384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:17:57.787693 14384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:17:57.787755 14384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:17:57.798494 14384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.173.148:35993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.173.190:33897,127.10.173.189:42919,127.10.173.188:37937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/krb5kdc/kudu_127.10.173.188.keytab
--principal=kudu/127.10.173.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.173.188:37937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.173.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 21:01:54 UTC on ccycloud.quasar-ltooxc.root.comops.site
build id 4581475
ASAN enabled
I20260811 22:17:57.799784 14384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:17:57.801066 14384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:17:57.813190 14398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:57.813283 14394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:17:57.813262 14396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:17:57.814296 14384 server_base.cc:1034] running on GCE node
I20260811 22:17:57.815119 14384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:17:57.816586 14384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:17:57.817804 14384 hybrid_clock.cc:648] HybridClock initialized: now 1786486677817726 us; error 52 us; skew 500 ppm
I20260811 22:17:57.818142 14384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:17:57.821290 14384 init.cc:377] Logged in from keytab as kudu/127.10.173.188@KRBTEST.COM (short username kudu)
I20260811 22:17:57.822837 14384 webserver.cc:460] Webserver started at http://127.10.173.188:44213/ using document root <none> and password file <none>
I20260811 22:17:57.823530 14384 fs_manager.cc:359] Metadata directory not provided
I20260811 22:17:57.823670 14384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:17:57.823997 14384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:17:57.826987 14384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/data/instance:
uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc"
format_stamp: "Formatted at 2026-08-11 22:17:57 on dist-test-slave-68xk"
server_key: "aeff102b43306f46efcbbdeb4027c4a1"
server_key_iv: "7fba07f84f5497743e3f680c3f0ce60f"
server_key_version: "encryptionkey@0"
I20260811 22:17:57.827776 14384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/wal/instance:
uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc"
format_stamp: "Formatted at 2026-08-11 22:17:57 on dist-test-slave-68xk"
server_key: "aeff102b43306f46efcbbdeb4027c4a1"
server_key_iv: "7fba07f84f5497743e3f680c3f0ce60f"
server_key_version: "encryptionkey@0"
I20260811 22:17:57.832975 14384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:17:57.836540 14406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:17:57.837862 14384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:17:57.838043 14384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/data,/tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/wal
uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc"
format_stamp: "Formatted at 2026-08-11 22:17:57 on dist-test-slave-68xk"
server_key: "aeff102b43306f46efcbbdeb4027c4a1"
server_key_iv: "7fba07f84f5497743e3f680c3f0ce60f"
server_key_version: "encryptionkey@0"
I20260811 22:17:57.838272 14384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:17:57.854184 14384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:17:57.858733 14384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:17:57.859031 14384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:17:57.876981 14384 rpc_server.cc:307] RPC server started. Bound to: 127.10.173.188:37937
I20260811 22:17:57.877002 14486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.173.188:37937 every 8 connection(s)
I20260811 22:17:57.878539 14384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/data/info.pb
I20260811 22:17:57.882525 14487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:17:57.887902 10934 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskZXazun/build/asan/bin/kudu as pid 14384
I20260811 22:17:57.888135 10934 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskZXazun/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486285256683-10934-0/minicluster-data/master-2/wal/instance
I20260811 22:17:57.888628 10934 external_mini_cluster.cc:1428] Setting key 84d53a01691a456cc5e197c16a0dee8b
I20260811 22:17:57.889075 14487 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
I20260811 22:17:57.909511 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.891922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.10.173.190:33897)
0811 22:17:57.892504 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:57.892512 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:57.893074 (+ 562us) server_negotiation.cc:407] Connection header received
0811 22:17:57.894170 (+ 1096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:57.894174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:57.894264 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:57.894395 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:57.895686 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.896407 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.898000 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.898533 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.898684 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.902694 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:57.902700 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:57.902705 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:57.902747 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:57.905809 (+ 3062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.906728 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.906732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.906735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.906876 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.907375 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.907380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.907384 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.907783 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:57.907949 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:57.908871 (+ 922us) server_negotiation.cc:299] Negotiation successful
0811 22:17:57.909044 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":312,"threads_started":1}
I20260811 22:17:57.910593 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.890551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33682 (local address 127.10.173.188:37937)
0811 22:17:57.891338 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:57.891372 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:57.891423 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:17:57.893776 (+ 2353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:57.893797 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:57.894034 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:57.894263 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:57.895770 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.896932 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.898292 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.898884 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.899027 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.903651 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:57.903665 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:57.903684 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:57.903735 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:57.907014 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.907952 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.907960 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.907963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.908171 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.908566 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.908570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.908573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.908994 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:57.909310 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:57.909696 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 22:17:57.909933 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":580,"thread_start_us":380,"threads_started":1}
I20260811 22:17:57.910636 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.891898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.190:33897 (local address 127.0.0.1:46788)
0811 22:17:57.892919 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.892955 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:57.893816 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:57.894490 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:57.894497 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:57.895141 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:57.895505 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.895521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.896593 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.896597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.897826 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.897839 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.898777 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.898784 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.899001 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.900012 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:57.900040 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:57.902467 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:57.905952 (+ 3485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.905967 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.905977 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.906591 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.906994 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.906998 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.907001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.907239 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.907962 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:57.907969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:57.908390 (+ 421us) client_negotiation.cc:769] Sending connection context
0811 22:17:57.908727 (+ 337us) client_negotiation.cc:240] Negotiation successful
0811 22:17:57.909224 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":145,"thread_start_us":574,"threads_started":1}
I20260811 22:17:57.913950 14487 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.190@KRBTEST.COM
I20260811 22:17:57.929167 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.914859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.189:42919 (local address 127.0.0.1:35192)
0811 22:17:57.915234 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.915252 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:57.915378 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:57.915704 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:57.915711 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:57.916053 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:57.916199 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.916206 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.917335 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.917341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.918677 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.918689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.919546 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.919550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.919870 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.920375 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:57.920400 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:57.923079 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:57.926720 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.926726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.926729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.927089 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.927482 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.927486 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.927490 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.927657 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.928186 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:57.928190 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:57.928337 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 22:17:57.928503 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 22:17:57.928799 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256}
I20260811 22:17:57.929206 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.914859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35192 (local address 127.10.173.189:42919)
0811 22:17:57.915077 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:57.915083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:57.915336 (+ 253us) server_negotiation.cc:407] Connection header received
0811 22:17:57.915492 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:57.915496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:57.915560 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:57.915638 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:57.916350 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.917177 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.918816 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.919361 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.919519 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.923275 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:57.923282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:57.923289 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:57.923341 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:57.926583 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.927210 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.927216 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.927220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.927361 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.927774 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.927779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.927784 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.928041 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:57.928223 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:57.928599 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 22:17:57.928816 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:17:57.929795 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.913615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.10.173.190:33897)
0811 22:17:57.913797 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:57.913808 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:57.913827 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:17:57.913947 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:57.913951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:57.914013 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:57.914090 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:57.915771 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.916591 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.918334 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.918841 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.918970 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.924194 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:57.924199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:57.924202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:57.924233 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:57.927132 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.927789 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.927795 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.927799 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.927934 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.928320 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.928325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.928329 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.928606 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:57.928893 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:57.929217 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 22:17:57.929456 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:17:57.930127 14258 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } attempt: 4
I20260811 22:17:57.930655 14487 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486676, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.190@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.188@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
Aug 11 22:17:57 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.173.189@KRBTEST.COM
I20260811 22:17:57.947921 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.931223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33690)
0811 22:17:57.931435 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.931460 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:57.931565 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:57.932408 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:57.932413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:57.932901 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:57.933133 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.933147 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.934936 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.934942 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.936383 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.936394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.937185 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.937191 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.937394 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.938001 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:57.938026 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:57.940782 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:57.945113 (+ 4331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.945118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.945123 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.945712 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.946186 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.946191 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.946195 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.946368 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.947015 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:57.947020 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:57.947202 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 22:17:57.947367 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 22:17:57.947647 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 22:17:57.948743 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.931890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33694 (local address 127.10.173.188:37937)
0811 22:17:57.932638 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:57.932645 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:57.932669 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:17:57.932717 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:57.932722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:57.932782 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:57.932881 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:57.933689 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.934666 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.936344 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.937110 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.937243 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.941013 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:57.941020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:57.941038 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:57.941076 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:57.944771 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.945426 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.945431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.945435 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.945569 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.945935 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.945941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.945944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.946330 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:57.946536 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:57.946990 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 22:17:57.948259 (+ 1269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":642,"thread_start_us":424,"threads_started":1}
I20260811 22:17:57.948798 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.931852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33690 (local address 127.10.173.188:37937)
0811 22:17:57.932054 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:57.932061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:57.932076 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:17:57.932113 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:57.932118 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:57.932215 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:57.933282 (+ 1067us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:57.933474 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.934744 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.936526 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.937031 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.937178 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.940992 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:57.940996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:57.941000 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:57.941027 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:57.944920 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.945892 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.945898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.945902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.946051 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.946536 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.946541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.946544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.946894 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:57.947034 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:57.947439 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 22:17:57.948401 (+ 962us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":828,"server-negotiator.queue_time_us":89,"spinlock_wait_cycles":32512}
I20260811 22:17:57.949375 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.931589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33694)
0811 22:17:57.931855 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:57.931873 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:57.931971 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:57.932951 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:57.932956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:57.933364 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:57.933519 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.933537 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.934792 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.934800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.936170 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:57.936187 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.937418 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.937423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:57.937641 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.938191 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:57.938212 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:57.940761 (+ 2549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:57.944902 (+ 4141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.944910 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.944914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.945302 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.945677 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:57.945683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:57.945687 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:57.945827 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:57.946504 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:57.946510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:57.946678 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 22:17:57.946846 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 22:17:57.948607 (+ 1761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":201}
I20260811 22:17:57.954921 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:57.932282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35208 (local address 127.10.173.189:42919)
0811 22:17:57.932445 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:57.932451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:57.932739 (+ 288us) server_negotiation.cc:407] Connection header received
0811 22:17:57.932882 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:57.932887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:57.932959 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:57.933056 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:57.934860 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.935604 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.937278 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:57.937947 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:57.938091 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:57.943665 (+ 5574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:57.943672 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:57.943676 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:57.943718 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:57.947405 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.948244 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.948249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.948253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.948427 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:57.948963 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:57.948966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:57.948968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:57.949321 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:57.950019 (+ 698us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:57.954324 (+ 4305us) server_negotiation.cc:299] Negotiation successful
0811 22:17:57.954666 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:17:57.955554 14258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133: Bootstrap starting.
I20260811 22:17:57.955992 14487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc: Bootstrap starting.
I20260811 22:17:57.959836 14258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:57.959834 14487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:57.959868 10934 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:17:57.961226 14258 log.cc:824] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:57.961226 14487 log.cc:824] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:17:57.964406 14487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc: No bootstrap required, opened a new log
I20260811 22:17:57.964771 14258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133: No bootstrap required, opened a new log
I20260811 22:17:57.962327 14216 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46790
I20260811 22:17:57.972713 14258 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } }
I20260811 22:17:57.973138 14258 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3f9b6f8f6c34491a22cc7c5fcef7133, State: Initialized, Role: FOLLOWER
I20260811 22:17:57.973654 14258 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } }
I20260811 22:17:57.974567 14487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } }
I20260811 22:17:57.974980 14487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d6c82b2d69b4e51ba9094dcab47a5bc, State: Initialized, Role: FOLLOWER
I20260811 22:17:57.974705 14506 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } } }
I20260811 22:17:57.975206 14506 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:57.975473 14487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } }
I20260811 22:17:57.976127 14258 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:57.976290 14216 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:17:57.976583 14216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:57.976464 14507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } } }
I20260811 22:17:57.976907 14507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:57.977651 14487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:57.979768 14216 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } }
I20260811 22:17:57.981555 14216 leader_election.cc:288] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [CANDIDATE]: Term 1 election: Requested vote from peers 17060a024cc54a5cbb1aecb6af6aa908 (127.10.173.189:42919), 5d6c82b2d69b4e51ba9094dcab47a5bc (127.10.173.188:37937)
I20260811 22:17:57.981945 14447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc"
I20260811 22:17:57.982309 14447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:57.984799 14447 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3f9b6f8f6c34491a22cc7c5fcef7133 in term 1.
I20260811 22:17:57.985487 14180 leader_election.cc:302] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d6c82b2d69b4e51ba9094dcab47a5bc, d3f9b6f8f6c34491a22cc7c5fcef7133; no voters:
I20260811 22:17:57.986030 14506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:17:57.992152 14506 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [term 1 LEADER]: Becoming Leader. State: Replica: d3f9b6f8f6c34491a22cc7c5fcef7133, State: Running, Role: LEADER
I20260811 22:17:57.992933 14506 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } }
I20260811 22:17:58.001705 14516 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3f9b6f8f6c34491a22cc7c5fcef7133. Latest consensus state: current_term: 1 leader_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } } }
I20260811 22:17:58.002230 14516 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:58.009269 14530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 22:17:58.025333 14184 tablet.cc:2369] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:17:58.026324 14530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:17:58.069761 14376 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } attempt: 5
Aug 11 22:17:58 dist-test-slave-68xk krb5kdc[14133](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486677, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.173.189@KRBTEST.COM for kudu/127.10.173.188@KRBTEST.COM
I20260811 22:17:58.090565 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.070670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.10.173.188:37937)
0811 22:17:58.071230 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.071238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.071256 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:17:58.071314 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.071319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.071393 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.071489 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.072602 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.073702 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.075806 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.076466 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.076614 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.081114 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:58.081124 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:58.081132 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:58.081184 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:58.085678 (+ 4494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:58.086800 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:58.086808 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:58.086813 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:58.086993 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:58.087937 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:58.087944 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:58.087951 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:58.088453 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:58.088634 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.089698 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.090023 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414}
I20260811 22:17:58.092979 14380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.070388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.173.188:37937 (local address 127.0.0.1:33712)
0811 22:17:58.070863 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 22:17:58.070883 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:17:58.070994 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:17:58.071527 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:17:58.071533 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:17:58.072166 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:17:58.072430 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:58.072447 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.073819 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.073828 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:58.075640 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:17:58.075659 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.076823 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.076831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:17:58.077066 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.077834 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:17:58.077860 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:17:58.080864 (+ 3004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:17:58.085912 (+ 5048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:58.085921 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:58.085928 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:58.086611 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:58.087236 (+ 625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:17:58.087244 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:17:58.087251 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:17:58.087663 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:17:58.088763 (+ 1100us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:17:58.088772 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:17:58.089153 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 22:17:58.091375 (+ 2222us) client_negotiation.cc:240] Negotiation successful
0811 22:17:58.092111 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346}
I20260811 22:17:58.096542 14447 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [term 1 FOLLOWER]: Refusing update from remote peer d3f9b6f8f6c34491a22cc7c5fcef7133: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:17:58.097647 14516 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:17:58.101572 14376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908: Bootstrap starting.
I20260811 22:17:58.102135 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.039120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35216 (local address 127.10.173.189:42919)
0811 22:17:58.039325 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.039333 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.083093 (+ 43760us) server_negotiation.cc:407] Connection header received
0811 22:17:58.083301 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.083306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.083388 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.083484 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.085867 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.087031 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.089169 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.089894 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.090002 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.093524 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:58.093530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:58.093537 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:58.093592 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:58.097922 (+ 4330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:58.098941 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:58.098946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:58.098950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:58.099167 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:58.099666 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:58.099671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:58.099674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:58.100506 (+ 832us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:58.100753 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.101504 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.101764 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:17:58.068073 14155 master_runner.cc:186] Error getting master registration for 127.10.173.189:42919: OK, instance_id { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" instance_seqno: 1786486677328002 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:17:58.105464 14155 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:17:58.107788 14376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908: Neither blocks nor log segments found. Creating new log.
I20260811 22:17:58.109206 14376 log.cc:824] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908: Log is configured to *not* fsync() on all Append() calls
W20260811 22:17:58.111994 14415 tablet.cc:2369] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:17:58.114060 14376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908: No bootstrap required, opened a new log
I20260811 22:17:58.125550 14376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } }
I20260811 22:17:58.125955 14376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17060a024cc54a5cbb1aecb6af6aa908, State: Initialized, Role: FOLLOWER
I20260811 22:17:58.126513 14376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } }
I20260811 22:17:58.127193 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.038579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46798 (local address 127.10.173.190:33897)
0811 22:17:58.038940 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.038950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.047004 (+ 8054us) server_negotiation.cc:407] Connection header received
0811 22:17:58.047358 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.047366 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.047475 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.047562 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.083510 (+ 35948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.084692 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.090056 (+ 5364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.090775 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.090892 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.110691 (+ 19799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:58.110700 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:58.110707 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:58.110756 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:58.116212 (+ 5456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:58.124400 (+ 8188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:58.124406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:58.124410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:58.124608 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:58.125120 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:58.125126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:58.125130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:58.125552 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:58.125862 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.126403 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.126789 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260811 22:17:58.128487 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.046679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33704 (local address 127.10.173.188:37937)
0811 22:17:58.046900 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.046909 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.087380 (+ 40471us) server_negotiation.cc:407] Connection header received
0811 22:17:58.087643 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.087649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.087727 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.087792 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.106320 (+ 18528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.107566 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.111464 (+ 3898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.112242 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.112352 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.119272 (+ 6920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:17:58.119282 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:17:58.119290 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:17:58.119347 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:17:58.123384 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:58.124319 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:58.124326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:58.124333 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:58.124676 (+ 343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:17:58.125692 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:17:58.125697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:17:58.125700 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:17:58.126121 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:17:58.126379 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.127507 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.127728 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:17:58.130606 14507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3f9b6f8f6c34491a22cc7c5fcef7133. Latest consensus state: current_term: 1 leader_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } } }
I20260811 22:17:58.130929 14507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:58.132560 14559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } } }
I20260811 22:17:58.133661 14559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:58.134795 14376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:17:58.135179 14337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:17:58.138882 14338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "17060a024cc54a5cbb1aecb6af6aa908"
I20260811 22:17:58.139464 14337 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [term 1 FOLLOWER]: Refusing update from remote peer d3f9b6f8f6c34491a22cc7c5fcef7133: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:17:58.140805 14543 mvcc.cc:205] Tried to move back new op lower bound from 7317449433339428864 to 7317449433066946560. Current Snapshot: MvccSnapshot[applied={T|T < 7317449433339428864}]
I20260811 22:17:58.142989 14551 mvcc.cc:205] Tried to move back new op lower bound from 7317449433339428864 to 7317449433066946560. Current Snapshot: MvccSnapshot[applied={T|T < 7317449433339428864}]
I20260811 22:17:58.143687 14530 catalog_manager.cc:1347] Generated new cluster ID: 7e0d1f35a0b842ae90a9441395c1a0ec
I20260811 22:17:58.143795 14530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:17:58.143613 14516 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } } }
I20260811 22:17:58.143939 14516 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:58.144320 14338 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3f9b6f8f6c34491a22cc7c5fcef7133 in term 1.
I20260811 22:17:58.145203 14506 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:17:58.145936 14507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } } }
I20260811 22:17:58.146159 14507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:58.165297 14530 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:17:58.167467 14530 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:17:58.151176 14384 master_runner.cc:186] Error getting master registration for 127.10.173.189:42919: OK, instance_id { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" instance_seqno: 1786486677328002 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:17:58.181324 14384 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:17:58.181116 14560 catalog_manager.cc:1259] Loaded cluster ID: 7e0d1f35a0b842ae90a9441395c1a0ec
I20260811 22:17:58.184015 14560 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc: loading cluster ID for follower catalog manager: success
I20260811 22:17:58.187517 14560 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc: acquiring CA information for follower catalog manager: success
W20260811 22:17:58.188723 14560 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260811 22:17:58.199854 14530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133: Generated new TSK 0
I20260811 22:17:58.201247 14530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:17:58.218310 14559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3f9b6f8f6c34491a22cc7c5fcef7133. Latest consensus state: current_term: 1 leader_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } } }
I20260811 22:17:58.218714 14559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:58.219424 14577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } } }
I20260811 22:17:58.219823 14577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:17:58.226715 14516 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3f9b6f8f6c34491a22cc7c5fcef7133" member_type: VOTER last_known_addr { host: "127.10.173.190" port: 33897 } } peers { permanent_uuid: "17060a024cc54a5cbb1aecb6af6aa908" member_type: VOTER last_known_addr { host: "127.10.173.189" port: 42919 } } peers { permanent_uuid: "5d6c82b2d69b4e51ba9094dcab47a5bc" member_type: VOTER last_known_addr { host: "127.10.173.188" port: 37937 } } }
I20260811 22:17:58.227761 14516 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3f9b6f8f6c34491a22cc7c5fcef7133 [sys.catalog]: This master's current role is: LEADER
I20260811 22:17:58.249462 14592 catalog_manager.cc:1259] Loaded cluster ID: 7e0d1f35a0b842ae90a9441395c1a0ec
I20260811 22:17:58.249655 14592 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908: loading cluster ID for follower catalog manager: success
I20260811 22:17:58.253564 14592 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908: acquiring CA information for follower catalog manager: success
I20260811 22:17:58.256269 14592 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 17060a024cc54a5cbb1aecb6af6aa908: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 22:17:58.360345 14381 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.341896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35224 (local address 127.10.173.189:42919)
0811 22:17:58.342138 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.342146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.356351 (+ 14205us) server_negotiation.cc:407] Connection header received
0811 22:17:58.358703 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.358710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.358832 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.358911 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.360108 (+ 1197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35224: BlockingRecv error: recv got EOF from 127.0.0.1:35224 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 22:17:58.363615 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.346293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.10.173.188:37937)
0811 22:17:58.346520 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.346527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.358324 (+ 11797us) server_negotiation.cc:407] Connection header received
0811 22:17:58.361666 (+ 3342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.361672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.361784 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.361860 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.363361 (+ 1501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33726: BlockingRecv error: recv got EOF from 127.0.0.1:33726 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 22:17:58.365289 14493 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.352441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46812 (local address 127.10.173.190:33897)
0811 22:17:58.352659 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.352666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.361241 (+ 8575us) server_negotiation.cc:407] Connection header received
0811 22:17:58.362087 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.362093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.362193 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.362261 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.365017 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46812: BlockingRecv error: recv got EOF from 127.0.0.1:46812 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:17:58.426127 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.406875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35240 (local address 127.10.173.189:42919)
0811 22:17:58.407086 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.407093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.417687 (+ 10594us) server_negotiation.cc:407] Connection header received
0811 22:17:58.418291 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.418299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.418566 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.418717 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:58.419502 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.420690 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.422833 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.423558 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.423658 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.424474 (+ 816us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:58.425260 (+ 786us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:58.425393 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.425652 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.425759 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 22:17:58.470700 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.460725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33738 (local address 127.10.173.188:37937)
0811 22:17:58.460921 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.460928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.468342 (+ 7414us) server_negotiation.cc:407] Connection header received
0811 22:17:58.468928 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.468936 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.469092 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.469174 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.470381 (+ 1207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33738: BlockingRecv error: recv got EOF from 127.0.0.1:33738 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:17:58.521212 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.502934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.10.173.189:42919)
0811 22:17:58.503134 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.503143 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.513817 (+ 10674us) server_negotiation.cc:407] Connection header received
0811 22:17:58.514443 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.514451 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.514594 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.514672 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:58.515414 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.516637 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.518764 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.519486 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.519592 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.520242 (+ 650us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:58.520413 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:58.520477 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.520770 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.520861 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 22:17:58.569128 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.558781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33742 (local address 127.10.173.188:37937)
0811 22:17:58.558985 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.558995 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.566945 (+ 7950us) server_negotiation.cc:407] Connection header received
0811 22:17:58.567478 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.567484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.567599 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.567680 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.568824 (+ 1144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33742: BlockingRecv error: recv got EOF from 127.0.0.1:33742 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:17:58.618162 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.602943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35254 (local address 127.10.173.189:42919)
0811 22:17:58.603144 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.603151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.610869 (+ 7718us) server_negotiation.cc:407] Connection header received
0811 22:17:58.611447 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.611455 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.611591 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.611673 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:58.612436 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.613566 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.615695 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.616442 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.616565 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.617228 (+ 663us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:58.617365 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:58.617432 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.617702 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.617809 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 22:17:58.665228 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.652923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33754 (local address 127.10.173.188:37937)
0811 22:17:58.653173 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.653182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.663085 (+ 9903us) server_negotiation.cc:407] Connection header received
0811 22:17:58.663393 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.663402 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.663554 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.663674 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.664905 (+ 1231us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33754: BlockingRecv error: recv got EOF from 127.0.0.1:33754 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 22:17:58.722548 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.703474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35262 (local address 127.10.173.189:42919)
0811 22:17:58.703804 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.703815 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.714244 (+ 10429us) server_negotiation.cc:407] Connection header received
0811 22:17:58.714860 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.714868 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.714983 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.715061 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:58.715914 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.717323 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.719923 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.720719 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.720833 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.721560 (+ 727us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:58.721735 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:58.721823 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.722109 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.722212 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
W20260811 22:17:58.779400 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.762704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33762 (local address 127.10.173.188:37937)
0811 22:17:58.763016 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.763027 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.777233 (+ 14206us) server_negotiation.cc:407] Connection header received
0811 22:17:58.777821 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.777828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.777933 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.778079 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.779129 (+ 1050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33762: BlockingRecv error: recv got EOF from 127.0.0.1:33762 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
I20260811 22:17:58.812496 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.792227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35268 (local address 127.10.173.189:42919)
0811 22:17:58.792481 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.792488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.803360 (+ 10872us) server_negotiation.cc:407] Connection header received
0811 22:17:58.803932 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.803942 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.804047 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.804135 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:58.804818 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.806078 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.809897 (+ 3819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.810725 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.810836 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.811566 (+ 730us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:58.811710 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:58.811779 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.812025 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.812134 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260811 22:17:58.826978 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.824597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33768 (local address 127.10.173.188:37937)
0811 22:17:58.824786 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.824792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.825410 (+ 618us) server_negotiation.cc:407] Connection header received
0811 22:17:58.825599 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.825605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.825702 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.825813 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.826738 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33768: BlockingRecv error: recv got EOF from 127.0.0.1:33768 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:17:58.852003 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.846917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35278 (local address 127.10.173.189:42919)
0811 22:17:58.847085 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.847089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.847491 (+ 402us) server_negotiation.cc:407] Connection header received
0811 22:17:58.847670 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.847674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.847745 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.847833 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:58.848213 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.848946 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.850245 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.850772 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.850874 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.851262 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:58.851377 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:58.851450 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.851578 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.851657 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 22:17:58.860296 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.858488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.10.173.188:37937)
0811 22:17:58.858790 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.858796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.859026 (+ 230us) server_negotiation.cc:407] Connection header received
0811 22:17:58.859170 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.859174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.859250 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.859346 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.860037 (+ 691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33782: BlockingRecv error: recv got EOF from 127.0.0.1:33782 (error 108)
Metrics: {"server-negotiator.queue_time_us":157}
I20260811 22:17:58.901979 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.896132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35280 (local address 127.10.173.189:42919)
0811 22:17:58.896439 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.896447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.896730 (+ 283us) server_negotiation.cc:407] Connection header received
0811 22:17:58.896992 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.896997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.897089 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.897256 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:58.897692 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.898465 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.899843 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.900281 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.900418 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.900952 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:58.901134 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:58.901256 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.901429 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.901597 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 22:17:58.909870 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.907567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33784 (local address 127.10.173.188:37937)
0811 22:17:58.907754 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.907758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.908312 (+ 554us) server_negotiation.cc:407] Connection header received
0811 22:17:58.908466 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.908470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.908564 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.908654 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.909563 (+ 909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33784: BlockingRecv error: recv got EOF from 127.0.0.1:33784 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:17:58.984270 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:58.978283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35288 (local address 127.10.173.189:42919)
0811 22:17:58.978557 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.978564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.978956 (+ 392us) server_negotiation.cc:407] Connection header received
0811 22:17:58.979241 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.979247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.979327 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.979444 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:58.979959 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.980798 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.982309 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:58.982861 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:58.983029 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:58.983487 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:58.983602 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:58.983711 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:58.983885 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 22:17:58.983974 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260811 22:17:58.993106 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:58.990929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33798 (local address 127.10.173.188:37937)
0811 22:17:58.991204 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:58.991210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:58.991546 (+ 336us) server_negotiation.cc:407] Connection header received
0811 22:17:58.991789 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:58.991793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:58.991936 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:58.992050 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:58.992867 (+ 817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33798: BlockingRecv error: recv got EOF from 127.0.0.1:33798 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 22:17:59.130779 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:59.125549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35294 (local address 127.10.173.189:42919)
0811 22:17:59.125797 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:59.125803 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:59.126149 (+ 346us) server_negotiation.cc:407] Connection header received
0811 22:17:59.126315 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:59.126320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:59.126396 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:59.126558 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:59.127012 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.127687 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.129120 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.129604 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.129704 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:59.130103 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:59.130205 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:59.130276 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:59.130469 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 22:17:59.130551 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 22:17:59.139192 14492 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:17:59.137206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.10.173.188:37937)
0811 22:17:59.137463 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:59.137471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:59.137730 (+ 259us) server_negotiation.cc:407] Connection header received
0811 22:17:59.137952 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:59.137958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:59.138040 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:59.138150 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:17:59.138989 (+ 839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33802: BlockingRecv error: recv got EOF from 127.0.0.1:33802 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:17:59.191496 14560 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5d6c82b2d69b4e51ba9094dcab47a5bc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:17:59.404699 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:59.399306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35304 (local address 127.10.173.189:42919)
0811 22:17:59.399573 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:59.399594 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:59.399835 (+ 241us) server_negotiation.cc:407] Connection header received
0811 22:17:59.400088 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:59.400093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:59.400166 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:59.400281 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:59.400751 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.401525 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.402943 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.403373 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.403517 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:59.403964 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:59.404080 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:59.404184 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:59.404345 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 22:17:59.404429 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:17:59.417135 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:59.410904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.10.173.188:37937)
0811 22:17:59.411148 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:59.411156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:59.411363 (+ 207us) server_negotiation.cc:407] Connection header received
0811 22:17:59.411624 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:59.411628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:59.411792 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:59.411929 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:59.412396 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.413165 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.414756 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.415231 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.415384 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:59.415926 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:59.416483 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:59.416675 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:59.416735 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 22:17:59.416827 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:17:59.434218 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:59.424097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35310 (local address 127.10.173.189:42919)
0811 22:17:59.424332 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:59.424341 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:59.426428 (+ 2087us) server_negotiation.cc:407] Connection header received
0811 22:17:59.426712 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:59.426733 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:59.426852 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:59.426930 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:59.428088 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.429519 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.431973 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.432659 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.432783 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:59.433198 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:59.433351 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:59.433453 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:59.433781 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 22:17:59.433878 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:17:59.435221 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:59.423602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33826 (local address 127.10.173.188:37937)
0811 22:17:59.423847 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:59.423856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:59.426100 (+ 2244us) server_negotiation.cc:407] Connection header received
0811 22:17:59.426662 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:59.426682 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:59.426801 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:59.426900 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:59.427986 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.429507 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.432640 (+ 3133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.433273 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.433412 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:59.434131 (+ 719us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:59.434258 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:59.434345 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:59.434851 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 22:17:59.434930 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:17:59.439040 14763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:17:59.423450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46826 (local address 127.10.173.190:33897)
0811 22:17:59.430901 (+ 7451us) server_negotiation.cc:206] Beginning negotiation
0811 22:17:59.430906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:17:59.432523 (+ 1617us) server_negotiation.cc:407] Connection header received
0811 22:17:59.432739 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:17:59.432746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:17:59.432953 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:17:59.433144 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:17:59.434032 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.435251 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.437010 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:17:59.437439 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:17:59.437539 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:17:59.438024 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:17:59.438487 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:17:59.438622 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:17:59.438743 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 22:17:59.438819 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7343,"thread_start_us":7235,"threads_started":1}
I20260811 22:17:59.444116 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 14155
I20260811 22:17:59.445179 14250 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:59.556187 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 14155
I20260811 22:17:59.588446 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 14267
I20260811 22:17:59.589752 14369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:59.701489 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 14267
I20260811 22:17:59.731333 10934 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskZXazun/build/asan/bin/kudu pid 14384
I20260811 22:17:59.732573 14479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 22:17:59.842133 10934 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskZXazun/build/asan/bin/kudu with pid 14384
2026-08-11T22:17:59Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5211 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10401 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (394519 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST